pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedEXPLAIN: showing ReadStream / prefetch stats
59+ messages / 6 participants
[nested] [flat]
* EXPLAIN: showing ReadStream / prefetch stats
@ 2026-03-15 19:49 Tomas Vondra <tomas@vondra.me>
2026-03-16 07:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-09-22 15:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
0 siblings, 3 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-03-15 19:49 UTC (permalink / raw)
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
Here's a patch adding prefetch/read-ahead info about IO to EXPLAIN. This
was initially developed as part of the index prefetch patches, but it
became clear it can be useful for other nodes using ReadStream. That
includes seqscan and bitmap heap scans, and (hopefully) will include
index scan in PG19.
The 0001 patch is "prerequisite" adjusting WaitReadBuffers to indicate
if the buffer read had to wait for the I/O. 0002 is the part this thread
is really about.
In EXPLAIN, this new info is enabled by a new option "IO", which adds
two new lines to the output:
--------------------
EXPLAIN (ANALYZE, COSTS off, TIMING off, IO)
SELECT * FROM t WHERE a < 100000;
QUERY PLAN
-------------------------------------------------
Seq Scan on t (actual rows=999996.00 loops=1)
Filter: (a < 100000)
Rows Removed by Filter: 4
Prefetch: avg=262.629 max=271 capacity=272
I/O: stalls=653 size=15.983 inprogress=15.934
Buffers: shared read=55556
Planning:
Buffers: shared hit=50 read=23
Planning Time: 7.358 ms
Execution Time: 358.214 ms
(10 rows)
--------------------
The first line "Prefetch" tracks the look-ahead distance, i.e. how many
blocks ahead the ReadStream is requesting.
The second line "I/O" is about the I/O requests actually issued - how
many times we had to wait for the block (when we get to process it),
average size of a request (in BLCKSZ blocks), and average number of
in-progress requests.
The above example shows we've been reading ~262 blocks ahead average,
with theoretical maximum of 272 (and at some point we read 271 blocks).
The average is pretty close to the maximum, so we've been reading ~2MB
ahead, which seems pretty good.
The I/O stats says we've been reading in 16-block requests, so 128kB
chunks (which aligns with io_combine_limit=16). And when issuing a new
I/O, there were ~16 requests in-progress already (which aligns with the
maximum distance, because 16*17=272).
We could track much more information. The WIP patches tracked histograms
of various metrics (distances, sizes, ...), and it was quite handy
during development. For regular EXPLAIN we chose to select only the most
useful subset, to make it meaningful and also cheap to collect. And also
generic enough to work for other table AM implementations.
Regarding the basic design, and a couple topics for review questions:
1) The information is collected by ReadStream, unconditionally.
I experimented with having a flag to "request" collecting these stats
(so that it can be done only for EXPLAIN ANALYZE), but that turned out
pretty useless. The stats are super cheap, so the extra flag did not
make a meaningful difference. And passing the flag to the scan/stream
was way too invasive. So all ReadStreams track it.
There's a ReadStreamInstrumentation defined in instrumet_node.h, but I'm
not convinced that's the right place. Maybe it should be defined in
read_stream.h instead.
2) This is inherently a TAM implementation detail.
The ReadStream is "inside" the TAM, and some TAMs may not even use a
ReadStream to do I/O. So EXPLAIN can't just inspect the stream directly,
that'd violate the layering. It needs to do it through the TAM API.
So this introduces a new TAM callback "scan_stats" which gets a scan
descriptor, and extracts stats from the stream (if any). The index
prefetching will need a second callback for IndexScanDesc.
It also introduces a "generic" TableScanStatsData struct, so that it
does not work with ReadStreamInstrumentation (because what if the TAM
does not use a ReadStream). Of course, for heap AM it's 1:1.
For heap AM the callback is pretty simple, it just collects the read
stream stats and "translates" it to the TableScanStats instance.
3) Adds shared instrumentation to SeqScan workers.
The patch also had to improve a parallel SeqScan to collect per-worker
instrumentation, similarly to a BitmapHeapScan. Until now this was not
needed, but it's needed for tracking per-worker prefetch stats.
Here's a couple questions I'm asking myself about this patch:
- Is the selected information really the minimal meaningful set of stats
we could track? Is it sufficiently clear what we're tracking, and/or
should we track something else?
- Is the TableScanStatsData generic enough? Can other TAMs (not using
ReadStream) use this to show meaningful stats?
- If the separation between TAM and the low-level instrumentation clear
enough? Or is the ReadStreamInstrumentation "leaking" somewhere? For
example, is it OK it's in SeqScanInstrumentation?
But I'm sure there are other questions I haven't thought of.
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v1-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch (2.8K, ../../a177a6dd-240b-455a-8f25-aca0b1c08c6e@vondra.me/2-v1-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch)
download | inline diff:
From 93df10e306368b02ccb00a160ab6adee8317c3c1 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH v1 1/2] bufmgr: Return whether WaitReadBuffers() needed to
wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 00bc609529a..c8ec7fb4f53 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1738,12 +1738,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1805,6 +1813,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1835,6 +1844,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1851,6 +1866,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4017896f951..a1ef04354dd 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -249,7 +249,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.53.0
[text/x-patch] v1-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE-VE.patch (34.7K, ../../a177a6dd-240b-455a-8f25-aca0b1c08c6e@vondra.me/3-v1-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE-VE.patch)
download | inline diff:
From 8b522d83b9eea412a4d01de2fea071926482b333 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Thu, 12 Mar 2026 02:04:07 +0100
Subject: [PATCH v1 2/2] explain: show prefetch stats in EXPLAIN (ANALYZE,
VERBOSE)
This adds details about AIO / prefetch for a number of executor nodes
using the ReadStream, notably:
- SeqScan
- BitmapHeapScan
The statistics is tracked by the ReadStream, and then propagated up
through the table AM interface.
The ReadStream tracks the statistics unconditionally, i.e. even outside
EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
integer counters), it's not worth gating this by a flag.
The TAM gets one new callback "scan_stats", to collect stats from the
scan (which fetch tuples from the TAM). There is also a new struct
TableScanStatsData/TableScanStats to separate the statistics from the
actual TAM implementation.
This required improving SeqScan to properly setup shared instrumentation
between the leader and workers, collect it at the end of a scan, etc.
---
doc/src/sgml/ref/explain.sgml | 12 +
src/backend/access/heap/heapam.c | 32 +++
src/backend/access/heap/heapam_handler.c | 1 +
src/backend/commands/explain.c | 262 ++++++++++++++++++++++
src/backend/commands/explain_state.c | 10 +
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeBitmapHeapscan.c | 16 ++
src/backend/executor/nodeSeqscan.c | 123 +++++++++-
src/backend/storage/aio/read_stream.c | 64 +++++-
src/include/access/heapam.h | 1 +
src/include/access/relscan.h | 27 +++
src/include/access/tableam.h | 17 ++
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 45 +++-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 2 +
src/include/storage/read_stream.h | 3 +
17 files changed, 615 insertions(+), 5 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 7dee77fd366..9da16c77d73 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -295,6 +296,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>FALSE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 8f1c11a9350..91ed8ce97c3 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1406,6 +1406,38 @@ heap_endscan(TableScanDesc sscan)
pfree(scan);
}
+/*
+ * heap_scan_stats
+ * return stats collected by the read stream
+ *
+ * Returns NULL if the scan is not using a read stream.
+ */
+TableScanStats
+heap_scan_stats(TableScanDesc sscan)
+{
+ HeapScanDesc scan = (HeapScanDesc) sscan;
+ ReadStreamInstrumentation stats;
+ TableScanStats res;
+
+ if (!scan->rs_read_stream)
+ return NULL;
+
+ stats = read_stream_prefetch_stats(scan->rs_read_stream);
+
+ res = palloc0(sizeof(TableScanStatsData));
+
+ res->prefetch_count = stats.prefetch_count;
+ res->distance_sum = stats.distance_sum;
+ res->distance_max = stats.distance_max;
+ res->distance_capacity = stats.distance_capacity;
+ res->stall_count = stats.stall_count;
+ res->io_count = stats.io_count;
+ res->io_nblocks = stats.io_nblocks;
+ res->io_in_progress = stats.io_in_progress;
+
+ return res;
+}
+
HeapTuple
heap_getnext(TableScanDesc sscan, ScanDirection direction)
{
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2d1ee9ac95d..1bf14b428e1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2641,6 +2641,7 @@ static const TableAmRoutine heapam_methods = {
.scan_set_tidrange = heap_set_tidrange,
.scan_getnextslot_tidrange = heap_getnextslot_tidrange,
+ .scan_stats = heap_scan_stats,
.parallelscan_estimate = table_block_parallelscan_estimate,
.parallelscan_initialize = table_block_parallelscan_initialize,
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 93918a223b8..8a86390c45b 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,8 @@
*/
#include "postgres.h"
+#include "access/genam.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -138,6 +140,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_info(ScanState *planstate,
+ ExplainState *es);
+static void show_worker_io_info(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2007,6 +2014,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_info((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2025,6 +2033,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_info((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2313,6 +2322,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+
+ /* show prefetch info for the given worker */
+ show_worker_io_info(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3983,6 +3996,255 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+/*
+ * show_scan_io_info
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_info(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ TableScanStats leader_stats;
+ TableScanStatsData stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started, no prefetch stats */
+ if (!(planstate && planstate->ss_currentScanDesc))
+ return;
+
+ /* collect prefetch statistics from the read stream */
+ leader_stats = table_scan_stats(planstate->ss_currentScanDesc);
+
+ if (leader_stats)
+ {
+ memcpy(&stats, leader_stats, sizeof(TableScanStatsData));
+ }
+ else
+ {
+ memset(&stats, 0, sizeof(TableScanStatsData));
+ }
+
+ /* Initialize counters with stats from the local process first */
+ switch (nodeTag(plan))
+ {
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ stats.prefetch_count += winstrument->stream.prefetch_count;
+ stats.distance_sum += winstrument->stream.distance_sum;
+ if (winstrument->stream.distance_max > stats.distance_max)
+ stats.distance_max = winstrument->stream.distance_max;
+ if (winstrument->stream.distance_capacity > stats.distance_capacity)
+ stats.distance_capacity = winstrument->stream.distance_capacity;
+ stats.stall_count += winstrument->stream.stall_count;
+ stats.io_count += winstrument->stream.io_count;
+ stats.io_nblocks += winstrument->stream.io_nblocks;
+ stats.io_in_progress += winstrument->stream.io_in_progress;
+ }
+ }
+
+ break;
+ }
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ stats.prefetch_count += winstrument->stream.prefetch_count;
+ stats.distance_sum += winstrument->stream.distance_sum;
+ if (winstrument->stream.distance_max > stats.distance_max)
+ stats.distance_max = winstrument->stream.distance_max;
+ if (winstrument->stream.distance_capacity > stats.distance_capacity)
+ stats.distance_capacity = winstrument->stream.distance_capacity;
+ stats.stall_count += winstrument->stream.stall_count;
+ stats.io_count += winstrument->stream.io_count;
+ stats.io_nblocks += winstrument->stream.io_nblocks;
+ stats.io_in_progress += winstrument->stream.io_in_progress;
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ /* don't print anything without prefetching */
+ if (stats.prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats.distance_sum * 1.0 / stats.prefetch_count),
+ stats.distance_max,
+ stats.distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats.stall_count > 0 || stats.io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats.stall_count);
+
+ if (stats.io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats.io_nblocks * 1.0 / stats.io_count),
+ (stats.io_in_progress * 1.0 / stats.io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats.distance_sum * 1.0 / stats.prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats.distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats.distance_capacity, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats.stall_count, es);
+
+ if (stats.io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats.io_nblocks * 1.0 / stats.io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats.io_in_progress * 1.0 / stats.io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+ }
+}
+
+/*
+ * show_io_worker_info
+ * show info about prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ ReadStreamInstrumentation *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stream;
+
+ break;
+ }
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stream;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->stall_count > 0 || stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats->stall_count);
+
+ if (stats->io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats->distance_capacity, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+
+ if (stats->io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+ }
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..f5cbdd21aaa 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -159,6 +159,10 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -185,6 +189,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ac84af294c9..ffc708ed6be 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1119,6 +1119,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 74eac93284e..171d85a45c2 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -275,6 +275,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
if (node->sinstrument != NULL && IsParallelWorker())
{
BitmapHeapScanInstrumentation *si;
+ TableScanStats stats;
Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
@@ -288,6 +289,21 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
+ {
+ si->stream.prefetch_count += stats->prefetch_count;
+ si->stream.distance_sum += stats->distance_sum;
+ if (stats->distance_max > si->stream.distance_max)
+ si->stream.distance_max = stats->distance_max;
+ if (stats->distance_capacity > si->stream.distance_capacity)
+ si->stream.distance_capacity = stats->distance_capacity;
+ si->stream.stall_count += stats->stall_count;
+ si->stream.io_count += stats->io_count;
+ si->stream.io_nblocks += stats->io_nblocks;
+ si->stream.io_in_progress += stats->io_in_progress;
+ }
}
/*
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index af3c788ce8b..5903f9b1b3c 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -294,6 +294,43 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+ TableScanStats stats;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
+ {
+ si->stream.prefetch_count += stats->prefetch_count;
+ si->stream.distance_sum += stats->distance_sum;
+ if (stats->distance_max > si->stream.distance_max)
+ si->stream.distance_max = stats->distance_max;
+ if (stats->distance_capacity > si->stream.distance_capacity)
+ si->stream.distance_capacity = stats->distance_capacity;
+ si->stream.stall_count += stats->stall_count;
+ si->stream.io_count += stats->io_count;
+ si->stream.io_nblocks += stats->io_nblocks;
+ si->stream.io_in_progress += stats->io_in_progress;
+ }
+ }
+
/*
* get information from node
*/
@@ -348,10 +385,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -367,14 +417,42 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -403,9 +481,48 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /*
+ * Workers don't get the pscan_len value in scan descriptor, so use the
+ * TAM callback again. The result has to match the earlier result in
+ * ExecSeqScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74ac..627afafef05 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -107,6 +108,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* stats counters */
+ ReadStreamInstrumentation stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -172,6 +176,38 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * read_stream_update_stats_prefetch
+ * update read_stream stats with current pinned buffer depth
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_update_stats_prefetch(ReadStream *stream)
+{
+ stream->stats.prefetch_count++;
+ stream->stats.distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stream->stats.distance_max)
+ stream->stats.distance_max = stream->pinned_buffers;
+}
+
+/*
+ * read_stream_update_stats_io
+ * update read_stream stats about size of I/O requests
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ stream->stats.io_count++;
+ stream->stats.io_nblocks += nblocks;
+ stream->stats.io_in_progress += in_progress;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +416,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -703,6 +742,10 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
+ /* zero the stats, then set capacity */
+ memset(&stream->stats, 0, sizeof(ReadStreamInstrumentation));
+ stream->stats.distance_capacity = max_pinned_buffers;
+
/*
* Skip the initial ramp-up phase if the caller says we're going to be
* reading the whole relation. This way we start out assuming we'll be
@@ -851,6 +894,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
@@ -860,6 +904,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* since we executed IO synchronously, count it as a stall */
+ stream->stats.stall_count += 1;
}
else
{
@@ -871,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
stream->fast_path = false;
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
#endif
@@ -916,12 +964,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we need to wait for IO */
+ if (needed_wait)
+ stream->stats.stall_count += 1;
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1034,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_update_stats_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
@@ -1118,3 +1173,10 @@ read_stream_end(ReadStream *stream)
read_stream_reset(stream);
pfree(stream);
}
+
+/* return the prefetch stats for the read_stream */
+ReadStreamInstrumentation
+read_stream_prefetch_stats(ReadStream *stream)
+{
+ return stream->stats;
+}
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ad993c07311..c8f428aece8 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -347,6 +347,7 @@ extern void heap_prepare_pagescan(TableScanDesc sscan);
extern void heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
bool allow_strat, bool allow_sync, bool allow_pagemode);
extern void heap_endscan(TableScanDesc sscan);
+extern TableScanStats heap_scan_stats(TableScanDesc sscan);
extern HeapTuple heap_getnext(TableScanDesc sscan, ScanDirection direction);
extern bool heap_getnextslot(TableScanDesc sscan,
ScanDirection direction, TupleTableSlot *slot);
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index ce340c076f8..cada9f28bd8 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -114,6 +114,33 @@ typedef struct ParallelBlockTableScanWorkerData
} ParallelBlockTableScanWorkerData;
typedef struct ParallelBlockTableScanWorkerData *ParallelBlockTableScanWorker;
+/*
+ * Generic prefetch stats for table scans.
+ */
+typedef struct TableScanStatsData
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} TableScanStatsData;
+typedef struct TableScanStatsData *TableScanStats;
+
/*
* Base class for fetches from a table via an index. This is the base-class
* for such scans, which needs to be embedded in the respective struct for
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..ab84b76443c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -380,6 +380,11 @@ typedef struct TableAmRoutine
ScanDirection direction,
TupleTableSlot *slot);
+ /*
+ * Collect statistics about table scan.
+ */
+ TableScanStats (*scan_stats) (TableScanDesc scan);
+
/* ------------------------------------------------------------------------
* Parallel table scan related functions.
* ------------------------------------------------------------------------
@@ -1006,6 +1011,18 @@ table_endscan(TableScanDesc scan)
scan->rs_rd->rd_tableam->scan_end(scan);
}
+/*
+ * Fetch statistics about table scan.
+ */
+static inline TableScanStats
+table_scan_stats(TableScanDesc scan)
+{
+ if (scan->rs_rd->rd_tableam->scan_stats)
+ return scan->rs_rd->rd_tableam->scan_stats(scan);
+
+ return NULL;
+}
+
/*
* Restart a relation scan.
*/
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 0b695f7d812..801e422437b 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -54,6 +54,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 8847d7f94fa..d94dbbc917d 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -41,9 +41,51 @@ typedef struct SharedAggInfo
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information about read streams
* ---------------------
*/
+typedef struct ReadStreamInstrumentation
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} ReadStreamInstrumentation;
+
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ ReadStreamInstrumentation stream;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +113,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ ReadStreamInstrumentation stream;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 63c067d5aae..c81d18d8aec 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1635,6 +1635,8 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SeqScanInstrumentation stats;
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..ec396db5369 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct ReadStreamInstrumentation ReadStreamInstrumentation;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -104,4 +105,6 @@ extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern ReadStreamInstrumentation read_stream_prefetch_stats(ReadStream *stream);
+
#endif /* READ_STREAM_H */
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-16 07:07 ` Lukas Fittl <lukas@fittl.com>
2026-03-16 17:29 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2 siblings, 1 reply; 59+ messages in thread
From: Lukas Fittl @ 2026-03-16 07:07 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Andres Freund <andres@anarazel.de>
Hi Tomas,
On Sun, Mar 15, 2026 at 12:49 PM Tomas Vondra <tomas@vondra.me> wrote:
> Here's a patch adding prefetch/read-ahead info about IO to EXPLAIN. This
> was initially developed as part of the index prefetch patches, but it
> became clear it can be useful for other nodes using ReadStream. That
> includes seqscan and bitmap heap scans, and (hopefully) will include
> index scan in PG19.
I think surfacing this to users makes a lot of sense - read streams
are hard to understand in detail, and this will make it easier to tie
back the I/O performance of a query to what happened in terms of
pre-fetching and stalls.
Big +1 on the concept of making this visible.
> In EXPLAIN, this new info is enabled by a new option "IO", which adds
> two new lines to the output:
I'm 50/50 if hiding this behind a new option really makes sense - if
its cheap enough to always capture, why not always show it?
e.g. we could consider doing with this what we did with BUFFERS
recently, which is to enable it by default. If someone finds that too
visually busy, they could still do IO OFF.
This also does make me wonder a bit what we should do about I/O
timings. Conceptually they'd belong closer to IO now than BUFFERS..
>
> --------------------
> EXPLAIN (ANALYZE, COSTS off, TIMING off, IO)
> SELECT * FROM t WHERE a < 100000;
>
> QUERY PLAN
> -------------------------------------------------
> Seq Scan on t (actual rows=999996.00 loops=1)
> Filter: (a < 100000)
> Rows Removed by Filter: 4
> Prefetch: avg=262.629 max=271 capacity=272
> I/O: stalls=653 size=15.983 inprogress=15.934
> Buffers: shared read=55556
> Planning:
> Buffers: shared hit=50 read=23
> Planning Time: 7.358 ms
> Execution Time: 358.214 ms
> (10 rows)
> --------------------
>
> The first line "Prefetch" tracks the look-ahead distance, i.e. how many
> blocks ahead the ReadStream is requesting.
>
> The second line "I/O" is about the I/O requests actually issued - how
> many times we had to wait for the block (when we get to process it),
> average size of a request (in BLCKSZ blocks), and average number of
> in-progress requests.
I wonder if we could somehow consolidate this into one line for the
text format? (specifically, moving prefetch into "I/O" at the end?)
I'm also not sure if "max" is really that useful, vs capacity?
> 1) The information is collected by ReadStream, unconditionally.
>
> I experimented with having a flag to "request" collecting these stats
> (so that it can be done only for EXPLAIN ANALYZE), but that turned out
> pretty useless. The stats are super cheap, so the extra flag did not
> make a meaningful difference. And passing the flag to the scan/stream
> was way too invasive. So all ReadStreams track it.
Makes sense. I did some testing with your patch against master, and
couldn't find any meaningful difference - its logical that this would
be cheap enough to always do.
> 2) This is inherently a TAM implementation detail.
>
> ...
>
> 3) Adds shared instrumentation to SeqScan workers.
>
> The patch also had to improve a parallel SeqScan to collect per-worker
> instrumentation, similarly to a BitmapHeapScan. Until now this was not
> needed, but it's needed for tracking per-worker prefetch stats.
I feel like something is off about the complexity of having each node
type ferry back the information. e.g. when you're implementing the
support for index prefetching, it'll require a bunch more changes. In
my mind, there is a reason we have a related problem that we solved
with the current pgBufferUsage, instead of dealing with that on a
per-node basis. I really feel we should have a more generic way of
dealing with this.
I'm saying that not being completely unbiased, because I think this
would be a great fit for the stack-based instrumentation I've been
discussing with Andres and Zsolt over at [0]. Andres at least
expressed some potential interest in getting that into 19, though its
definitely not a trivial patch set.
If we were to get stack-based instrumentation in, we could easily add
a new "IOUsage" to the Instrumentation struct, avoid any modification
to the TAM interface, and align it with how we treat BufferUsage and
WALUsage.
I've attached a prototype of how that could look like (apply the other
patch set first, v8, see commit fest entry [1] - also attached a
preparatory refactoring of using "Instrumentation" for parallel query
reporting, which avoids having individual structs there). Performance
is actually better for "EXPLAIN (ANALYZE, IO, BUFFERS OFF, TIMING
OFF)" with that patch set on a quick COUNT(*) test, but that's mostly
due to the overhead of ExecProcNode dispatching that I fixed in the
other patch set (v8/0006). I think otherwise this would be similar in
performance thanks to the stack avoiding any "accum diff" logic.
Its also worth noting that this would make it trivial to output this
information for utility commands that have read stream support, or
show aggregate statistics in pg_stat_statements/etc.
Let me know if that's at all of interest and happy to pair up on this
to make it work.
Thanks,
Lukas
[0]: https://www.postgresql.org/message-id/flat/CAP53PkzdBK8VJ1fS4AZ481LgMN8f9mJiC39ZRHqkFUSYq6KWmg%40mai...
[1]: https://commitfest.postgresql.org/patch/6023/
--
Lukas Fittl
Attachments:
[text/x-patch] nocfbot-0002-Use-Instrumentation-struct-for-parallel-wor.patch (29.2K, ../../CAP53Pkx_x2WUtM40r6+GZMVCW5d2PE8KKYQc1=ArODn0-WYq+A@mail.gmail.com/2-nocfbot-0002-Use-Instrumentation-struct-for-parallel-wor.patch)
download | inline diff:
From 418d18fd09930bcd89dccba7e9fe5ce3322184a4 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Sun, 15 Mar 2026 21:44:58 -0700
Subject: [PATCH vnocfbot 2/3] Use Instrumentation struct for parallel worker
communication
This simplifies the allocations a bit, since we don't need to separately
allocate WAL and buffer usage, and allows the easier addition of a
planned third struct in Instrumentation.
Author: Lukas Fittl <lukas@fittl.com>
Reviewed-by:
Discussion: https://postgr.es/m/
---
src/backend/access/brin/brin.c | 43 ++++++-----------
src/backend/access/gin/gininsert.c | 43 ++++++-----------
src/backend/access/nbtree/nbtsort.c | 43 ++++++-----------
src/backend/commands/vacuumparallel.c | 52 ++++++++-------------
src/backend/executor/execParallel.c | 66 ++++++++++++---------------
src/backend/executor/instrument.c | 14 +++---
src/include/executor/execParallel.h | 5 +-
src/include/executor/instrument.h | 4 +-
8 files changed, 99 insertions(+), 171 deletions(-)
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index d62eb7dee9..8b156365ee 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -49,8 +49,7 @@
#define PARALLEL_KEY_BRIN_SHARED UINT64CONST(0xB000000000000001)
#define PARALLEL_KEY_TUPLESORT UINT64CONST(0xB000000000000002)
#define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xB000000000000003)
-#define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xB000000000000004)
-#define PARALLEL_KEY_BUFFER_USAGE UINT64CONST(0xB000000000000005)
+#define PARALLEL_KEY_INSTRUMENTATION UINT64CONST(0xB000000000000004)
/*
* Status for index builds performed in parallel. This is allocated in a
@@ -146,8 +145,7 @@ typedef struct BrinLeader
BrinShared *brinshared;
Sharedsort *sharedsort;
Snapshot snapshot;
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *instr;
} BrinLeader;
/*
@@ -2385,8 +2383,7 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
BrinShared *brinshared;
Sharedsort *sharedsort;
BrinLeader *brinleader = palloc0_object(BrinLeader);
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *instr;
bool leaderparticipates = true;
int querylen;
@@ -2428,18 +2425,14 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
shm_toc_estimate_keys(&pcxt->estimator, 2);
/*
- * Estimate space for WalUsage and BufferUsage -- PARALLEL_KEY_WAL_USAGE
- * and PARALLEL_KEY_BUFFER_USAGE.
+ * Estimate space for Instrumentation -- PARALLEL_KEY_INSTRUMENTATION.
*
* If there are no extensions loaded that care, we could skip this. We
* have no way of knowing whether anyone's looking at instrumentation, so
* do it unconditionally.
*/
shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_estimate_keys(&pcxt->estimator, 1);
- shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
shm_toc_estimate_keys(&pcxt->estimator, 1);
/* Finally, estimate PARALLEL_KEY_QUERY_TEXT space */
@@ -2512,15 +2505,12 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
}
/*
- * Allocate space for each worker's WalUsage and BufferUsage; no need to
+ * Allocate space for each worker's Instrumentation; no need to
* initialize.
*/
- walusage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_WAL_USAGE, walusage);
- bufferusage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_BUFFER_USAGE, bufferusage);
+ instr = shm_toc_allocate(pcxt->toc,
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_INSTRUMENTATION, instr);
/* Launch workers, saving status for leader/caller */
LaunchParallelWorkers(pcxt);
@@ -2531,8 +2521,7 @@ _brin_begin_parallel(BrinBuildState *buildstate, Relation heap, Relation index,
brinleader->brinshared = brinshared;
brinleader->sharedsort = sharedsort;
brinleader->snapshot = snapshot;
- brinleader->walusage = walusage;
- brinleader->bufferusage = bufferusage;
+ brinleader->instr = instr;
/* If no workers were successfully launched, back out (do serial build) */
if (pcxt->nworkers_launched == 0)
@@ -2571,7 +2560,7 @@ _brin_end_parallel(BrinLeader *brinleader, BrinBuildState *state)
* or we might get incomplete data.)
*/
for (i = 0; i < brinleader->pcxt->nworkers_launched; i++)
- InstrAccumParallelQuery(&brinleader->bufferusage[i], &brinleader->walusage[i]);
+ InstrAccumParallelQuery(&brinleader->instr[i]);
/* Free last reference to MVCC snapshot, if one was used */
if (IsMVCCSnapshot(brinleader->snapshot))
@@ -2885,8 +2874,7 @@ _brin_parallel_build_main(dsm_segment *seg, shm_toc *toc)
LOCKMODE heapLockmode;
LOCKMODE indexLockmode;
QueryInstrumentation *instr;
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *worker_instr;
int sortmem;
/*
@@ -2947,11 +2935,8 @@ _brin_parallel_build_main(dsm_segment *seg, shm_toc *toc)
heapRel, indexRel, sortmem, false);
/* Report WAL/buffer usage during parallel execution */
- bufferusage = shm_toc_lookup(toc, PARALLEL_KEY_BUFFER_USAGE, false);
- walusage = shm_toc_lookup(toc, PARALLEL_KEY_WAL_USAGE, false);
- InstrEndParallelQuery(instr,
- &bufferusage[ParallelWorkerNumber],
- &walusage[ParallelWorkerNumber]);
+ worker_instr = shm_toc_lookup(toc, PARALLEL_KEY_INSTRUMENTATION, false);
+ InstrEndParallelQuery(instr, &worker_instr[ParallelWorkerNumber]);
index_close(indexRel, indexLockmode);
table_close(heapRel, heapLockmode);
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index 8cdcd2a9be..db8235944b 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -42,8 +42,7 @@
#define PARALLEL_KEY_GIN_SHARED UINT64CONST(0xB000000000000001)
#define PARALLEL_KEY_TUPLESORT UINT64CONST(0xB000000000000002)
#define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xB000000000000003)
-#define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xB000000000000004)
-#define PARALLEL_KEY_BUFFER_USAGE UINT64CONST(0xB000000000000005)
+#define PARALLEL_KEY_INSTRUMENTATION UINT64CONST(0xB000000000000004)
/*
* Status for index builds performed in parallel. This is allocated in a
@@ -135,8 +134,7 @@ typedef struct GinLeader
GinBuildShared *ginshared;
Sharedsort *sharedsort;
Snapshot snapshot;
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *instr;
} GinLeader;
typedef struct
@@ -942,8 +940,7 @@ _gin_begin_parallel(GinBuildState *buildstate, Relation heap, Relation index,
GinBuildShared *ginshared;
Sharedsort *sharedsort;
GinLeader *ginleader = palloc0_object(GinLeader);
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *instr;
bool leaderparticipates = true;
int querylen;
@@ -984,18 +981,14 @@ _gin_begin_parallel(GinBuildState *buildstate, Relation heap, Relation index,
shm_toc_estimate_keys(&pcxt->estimator, 2);
/*
- * Estimate space for WalUsage and BufferUsage -- PARALLEL_KEY_WAL_USAGE
- * and PARALLEL_KEY_BUFFER_USAGE.
+ * Estimate space for Instrumentation -- PARALLEL_KEY_INSTRUMENTATION.
*
* If there are no extensions loaded that care, we could skip this. We
* have no way of knowing whether anyone's looking at instrumentation, so
* do it unconditionally.
*/
shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_estimate_keys(&pcxt->estimator, 1);
- shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
shm_toc_estimate_keys(&pcxt->estimator, 1);
/* Finally, estimate PARALLEL_KEY_QUERY_TEXT space */
@@ -1063,15 +1056,12 @@ _gin_begin_parallel(GinBuildState *buildstate, Relation heap, Relation index,
}
/*
- * Allocate space for each worker's WalUsage and BufferUsage; no need to
+ * Allocate space for each worker's Instrumentation; no need to
* initialize.
*/
- walusage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_WAL_USAGE, walusage);
- bufferusage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_BUFFER_USAGE, bufferusage);
+ instr = shm_toc_allocate(pcxt->toc,
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_INSTRUMENTATION, instr);
/* Launch workers, saving status for leader/caller */
LaunchParallelWorkers(pcxt);
@@ -1082,8 +1072,7 @@ _gin_begin_parallel(GinBuildState *buildstate, Relation heap, Relation index,
ginleader->ginshared = ginshared;
ginleader->sharedsort = sharedsort;
ginleader->snapshot = snapshot;
- ginleader->walusage = walusage;
- ginleader->bufferusage = bufferusage;
+ ginleader->instr = instr;
/* If no workers were successfully launched, back out (do serial build) */
if (pcxt->nworkers_launched == 0)
@@ -1122,7 +1111,7 @@ _gin_end_parallel(GinLeader *ginleader, GinBuildState *state)
* or we might get incomplete data.)
*/
for (i = 0; i < ginleader->pcxt->nworkers_launched; i++)
- InstrAccumParallelQuery(&ginleader->bufferusage[i], &ginleader->walusage[i]);
+ InstrAccumParallelQuery(&ginleader->instr[i]);
/* Free last reference to MVCC snapshot, if one was used */
if (IsMVCCSnapshot(ginleader->snapshot))
@@ -2115,8 +2104,7 @@ _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc)
LOCKMODE heapLockmode;
LOCKMODE indexLockmode;
QueryInstrumentation *instr;
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *worker_instr;
int sortmem;
/*
@@ -2196,11 +2184,8 @@ _gin_parallel_build_main(dsm_segment *seg, shm_toc *toc)
heapRel, indexRel, sortmem, false);
/* Report WAL/buffer usage during parallel execution */
- bufferusage = shm_toc_lookup(toc, PARALLEL_KEY_BUFFER_USAGE, false);
- walusage = shm_toc_lookup(toc, PARALLEL_KEY_WAL_USAGE, false);
- InstrEndParallelQuery(instr,
- &bufferusage[ParallelWorkerNumber],
- &walusage[ParallelWorkerNumber]);
+ worker_instr = shm_toc_lookup(toc, PARALLEL_KEY_INSTRUMENTATION, false);
+ InstrEndParallelQuery(instr, &worker_instr[ParallelWorkerNumber]);
index_close(indexRel, indexLockmode);
table_close(heapRel, heapLockmode);
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index dfe4fd9459..c56da36023 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -65,8 +65,7 @@
#define PARALLEL_KEY_TUPLESORT UINT64CONST(0xA000000000000002)
#define PARALLEL_KEY_TUPLESORT_SPOOL2 UINT64CONST(0xA000000000000003)
#define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xA000000000000004)
-#define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xA000000000000005)
-#define PARALLEL_KEY_BUFFER_USAGE UINT64CONST(0xA000000000000006)
+#define PARALLEL_KEY_INSTRUMENTATION UINT64CONST(0xA000000000000005)
/*
* DISABLE_LEADER_PARTICIPATION disables the leader's participation in
@@ -194,8 +193,7 @@ typedef struct BTLeader
Sharedsort *sharedsort;
Sharedsort *sharedsort2;
Snapshot snapshot;
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *instr;
} BTLeader;
/*
@@ -1407,8 +1405,7 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
Sharedsort *sharedsort2;
BTSpool *btspool = buildstate->spool;
BTLeader *btleader = palloc0_object(BTLeader);
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *instr;
bool leaderparticipates = true;
int querylen;
@@ -1461,18 +1458,14 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
}
/*
- * Estimate space for WalUsage and BufferUsage -- PARALLEL_KEY_WAL_USAGE
- * and PARALLEL_KEY_BUFFER_USAGE.
+ * Estimate space for Instrumentation -- PARALLEL_KEY_INSTRUMENTATION.
*
* If there are no extensions loaded that care, we could skip this. We
* have no way of knowing whether anyone's looking at instrumentation, so
* do it unconditionally.
*/
shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_estimate_keys(&pcxt->estimator, 1);
- shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
shm_toc_estimate_keys(&pcxt->estimator, 1);
/* Finally, estimate PARALLEL_KEY_QUERY_TEXT space */
@@ -1559,15 +1552,12 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
}
/*
- * Allocate space for each worker's WalUsage and BufferUsage; no need to
+ * Allocate space for each worker's Instrumentation; no need to
* initialize.
*/
- walusage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_WAL_USAGE, walusage);
- bufferusage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_BUFFER_USAGE, bufferusage);
+ instr = shm_toc_allocate(pcxt->toc,
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_INSTRUMENTATION, instr);
/* Launch workers, saving status for leader/caller */
LaunchParallelWorkers(pcxt);
@@ -1579,8 +1569,7 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
btleader->sharedsort = sharedsort;
btleader->sharedsort2 = sharedsort2;
btleader->snapshot = snapshot;
- btleader->walusage = walusage;
- btleader->bufferusage = bufferusage;
+ btleader->instr = instr;
/* If no workers were successfully launched, back out (do serial build) */
if (pcxt->nworkers_launched == 0)
@@ -1619,7 +1608,7 @@ _bt_end_parallel(BTLeader *btleader)
* or we might get incomplete data.)
*/
for (i = 0; i < btleader->pcxt->nworkers_launched; i++)
- InstrAccumParallelQuery(&btleader->bufferusage[i], &btleader->walusage[i]);
+ InstrAccumParallelQuery(&btleader->instr[i]);
/* Free last reference to MVCC snapshot, if one was used */
if (IsMVCCSnapshot(btleader->snapshot))
@@ -1753,8 +1742,7 @@ _bt_parallel_build_main(dsm_segment *seg, shm_toc *toc)
LOCKMODE heapLockmode;
LOCKMODE indexLockmode;
QueryInstrumentation *instr;
- WalUsage *walusage;
- BufferUsage *bufferusage;
+ Instrumentation *worker_instr;
int sortmem;
#ifdef BTREE_BUILD_STATS
@@ -1836,11 +1824,8 @@ _bt_parallel_build_main(dsm_segment *seg, shm_toc *toc)
sharedsort2, sortmem, false);
/* Report WAL/buffer usage during parallel execution */
- bufferusage = shm_toc_lookup(toc, PARALLEL_KEY_BUFFER_USAGE, false);
- walusage = shm_toc_lookup(toc, PARALLEL_KEY_WAL_USAGE, false);
- InstrEndParallelQuery(instr,
- &bufferusage[ParallelWorkerNumber],
- &walusage[ParallelWorkerNumber]);
+ worker_instr = shm_toc_lookup(toc, PARALLEL_KEY_INSTRUMENTATION, false);
+ InstrEndParallelQuery(instr, &worker_instr[ParallelWorkerNumber]);
#ifdef BTREE_BUILD_STATS
if (log_btree_build_stats)
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index 75074fe4ef..753dd965d7 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -47,9 +47,8 @@
*/
#define PARALLEL_VACUUM_KEY_SHARED 1
#define PARALLEL_VACUUM_KEY_QUERY_TEXT 2
-#define PARALLEL_VACUUM_KEY_BUFFER_USAGE 3
-#define PARALLEL_VACUUM_KEY_WAL_USAGE 4
-#define PARALLEL_VACUUM_KEY_INDEX_STATS 5
+#define PARALLEL_VACUUM_KEY_INSTRUMENTATION 3
+#define PARALLEL_VACUUM_KEY_INDEX_STATS 4
/*
* Shared information among parallel workers. So this is allocated in the DSM
@@ -188,11 +187,8 @@ struct ParallelVacuumState
/* Shared dead items space among parallel vacuum workers */
TidStore *dead_items;
- /* Points to buffer usage area in DSM */
- BufferUsage *buffer_usage;
-
- /* Points to WAL usage area in DSM */
- WalUsage *wal_usage;
+ /* Points to instrumentation area in DSM */
+ Instrumentation *instr;
/*
* False if the index is totally unsuitable target for all parallel
@@ -250,8 +246,7 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
PVShared *shared;
TidStore *dead_items;
PVIndStats *indstats;
- BufferUsage *buffer_usage;
- WalUsage *wal_usage;
+ Instrumentation *instr;
bool *will_parallel_vacuum;
Size est_indstats_len;
Size est_shared_len;
@@ -304,18 +299,15 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
shm_toc_estimate_keys(&pcxt->estimator, 1);
/*
- * Estimate space for BufferUsage and WalUsage --
- * PARALLEL_VACUUM_KEY_BUFFER_USAGE and PARALLEL_VACUUM_KEY_WAL_USAGE.
+ * Estimate space for Instrumentation --
+ * PARALLEL_VACUUM_KEY_INSTRUMENTATION.
*
* If there are no extensions loaded that care, we could skip this. We
* have no way of knowing whether anyone's looking at instrumentation, so
* do it unconditionally.
*/
shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_estimate_keys(&pcxt->estimator, 1);
- shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
shm_toc_estimate_keys(&pcxt->estimator, 1);
/* Finally, estimate PARALLEL_VACUUM_KEY_QUERY_TEXT space */
@@ -396,17 +388,13 @@ parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes,
pvs->shared = shared;
/*
- * Allocate space for each worker's BufferUsage and WalUsage; no need to
- * initialize
+ * Allocate space for each worker's Instrumentation; no need to
+ * initialize.
*/
- buffer_usage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_VACUUM_KEY_BUFFER_USAGE, buffer_usage);
- pvs->buffer_usage = buffer_usage;
- wal_usage = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_VACUUM_KEY_WAL_USAGE, wal_usage);
- pvs->wal_usage = wal_usage;
+ instr = shm_toc_allocate(pcxt->toc,
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
+ shm_toc_insert(pcxt->toc, PARALLEL_VACUUM_KEY_INSTRUMENTATION, instr);
+ pvs->instr = instr;
/* Store query string for workers */
if (debug_query_string)
@@ -738,7 +726,7 @@ parallel_vacuum_process_all_indexes(ParallelVacuumState *pvs, int num_index_scan
WaitForParallelWorkersToFinish(pvs->pcxt);
for (int i = 0; i < pvs->pcxt->nworkers_launched; i++)
- InstrAccumParallelQuery(&pvs->buffer_usage[i], &pvs->wal_usage[i]);
+ InstrAccumParallelQuery(&pvs->instr[i]);
}
/*
@@ -996,8 +984,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
PVShared *shared;
TidStore *dead_items;
QueryInstrumentation *instr;
- BufferUsage *buffer_usage;
- WalUsage *wal_usage;
+ Instrumentation *worker_instr;
int nindexes;
char *sharedquery;
ErrorContextCallback errcallback;
@@ -1091,11 +1078,8 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
parallel_vacuum_process_safe_indexes(&pvs);
/* Report buffer/WAL usage during parallel execution */
- buffer_usage = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_BUFFER_USAGE, false);
- wal_usage = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_WAL_USAGE, false);
- InstrEndParallelQuery(instr,
- &buffer_usage[ParallelWorkerNumber],
- &wal_usage[ParallelWorkerNumber]);
+ worker_instr = shm_toc_lookup(toc, PARALLEL_VACUUM_KEY_INSTRUMENTATION, false);
+ InstrEndParallelQuery(instr, &worker_instr[ParallelWorkerNumber]);
/* Report any remaining cost-based vacuum delay time */
if (track_cost_delay_timing)
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 73534fa6c7..ebab6bc165 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -60,13 +60,12 @@
#define PARALLEL_KEY_EXECUTOR_FIXED UINT64CONST(0xE000000000000001)
#define PARALLEL_KEY_PLANNEDSTMT UINT64CONST(0xE000000000000002)
#define PARALLEL_KEY_PARAMLISTINFO UINT64CONST(0xE000000000000003)
-#define PARALLEL_KEY_BUFFER_USAGE UINT64CONST(0xE000000000000004)
+#define PARALLEL_KEY_INSTRUMENTATION UINT64CONST(0xE000000000000004)
#define PARALLEL_KEY_TUPLE_QUEUE UINT64CONST(0xE000000000000005)
-#define PARALLEL_KEY_INSTRUMENTATION UINT64CONST(0xE000000000000006)
+#define PARALLEL_KEY_NODE_INSTRUMENTATION UINT64CONST(0xE000000000000006)
#define PARALLEL_KEY_DSA UINT64CONST(0xE000000000000007)
#define PARALLEL_KEY_QUERY_TEXT UINT64CONST(0xE000000000000008)
#define PARALLEL_KEY_JIT_INSTRUMENTATION UINT64CONST(0xE000000000000009)
-#define PARALLEL_KEY_WAL_USAGE UINT64CONST(0xE00000000000000A)
#define PARALLEL_TUPLE_QUEUE_SIZE 65536
@@ -625,8 +624,6 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
char *pstmt_data;
char *pstmt_space;
char *paramlistinfo_space;
- BufferUsage *bufusage_space;
- WalUsage *walusage_space;
SharedExecutorInstrumentation *instrumentation = NULL;
SharedJitInstrumentation *jit_instrumentation = NULL;
int pstmt_len;
@@ -690,21 +687,14 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
shm_toc_estimate_keys(&pcxt->estimator, 1);
/*
- * Estimate space for BufferUsage.
+ * Estimate space for Instrumentation.
*
* If EXPLAIN is not in use and there are no extensions loaded that care,
* we could skip this. But we have no way of knowing whether anyone's
* looking at instrumentation, so do it unconditionally.
*/
shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_estimate_keys(&pcxt->estimator, 1);
-
- /*
- * Same thing for WalUsage.
- */
- shm_toc_estimate_chunk(&pcxt->estimator,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
shm_toc_estimate_keys(&pcxt->estimator, 1);
/* Estimate space for tuple queues. */
@@ -790,17 +780,18 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
shm_toc_insert(pcxt->toc, PARALLEL_KEY_PARAMLISTINFO, paramlistinfo_space);
SerializeParamList(estate->es_param_list_info, ¶mlistinfo_space);
- /* Allocate space for each worker's BufferUsage; no need to initialize. */
- bufusage_space = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(BufferUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_BUFFER_USAGE, bufusage_space);
- pei->buffer_usage = bufusage_space;
+ /*
+ * Allocate space for each worker's Instrumentation; no need to
+ * initialize.
+ */
+ {
+ Instrumentation *instr;
- /* Same for WalUsage. */
- walusage_space = shm_toc_allocate(pcxt->toc,
- mul_size(sizeof(WalUsage), pcxt->nworkers));
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_WAL_USAGE, walusage_space);
- pei->wal_usage = walusage_space;
+ instr = shm_toc_allocate(pcxt->toc,
+ mul_size(sizeof(Instrumentation), pcxt->nworkers));
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_INSTRUMENTATION, instr);
+ pei->instrumentation = instr;
+ }
/* Set up the tuple queues that the workers will write into. */
pei->tqueue = ExecParallelSetupTupleQueues(pcxt, false);
@@ -826,9 +817,9 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
instrument = GetInstrumentationArray(instrumentation);
for (i = 0; i < nworkers * e.nnodes; ++i)
InstrInitNode(&instrument[i], estate->es_instrument);
- shm_toc_insert(pcxt->toc, PARALLEL_KEY_INSTRUMENTATION,
+ shm_toc_insert(pcxt->toc, PARALLEL_KEY_NODE_INSTRUMENTATION,
instrumentation);
- pei->instrumentation = instrumentation;
+ pei->node_instrumentation = instrumentation;
if (estate->es_jit_flags != PGJIT_NONE)
{
@@ -1230,7 +1221,7 @@ ExecParallelFinish(ParallelExecutorInfo *pei)
* finish, or we might get incomplete data.)
*/
for (i = 0; i < nworkers; i++)
- InstrAccumParallelQuery(&pei->buffer_usage[i], &pei->wal_usage[i]);
+ InstrAccumParallelQuery(&pei->instrumentation[i]);
pei->finished = true;
}
@@ -1244,10 +1235,10 @@ ExecParallelFinish(ParallelExecutorInfo *pei)
void
ExecParallelCleanup(ParallelExecutorInfo *pei)
{
- /* Accumulate instrumentation, if any. */
- if (pei->instrumentation)
+ /* Accumulate node instrumentation, if any. */
+ if (pei->node_instrumentation)
ExecParallelRetrieveInstrumentation(pei->planstate,
- pei->instrumentation);
+ pei->node_instrumentation);
/* Accumulate JIT instrumentation, if any. */
if (pei->jit_instrumentation)
@@ -1471,8 +1462,6 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
{
FixedParallelExecutorState *fpes;
QueryInstrumentation *instr;
- BufferUsage *buffer_usage;
- WalUsage *wal_usage;
DestReceiver *receiver;
QueryDesc *queryDesc;
SharedExecutorInstrumentation *instrumentation;
@@ -1487,7 +1476,7 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
/* Set up DestReceiver, SharedExecutorInstrumentation, and QueryDesc. */
receiver = ExecParallelGetReceiver(seg, toc);
- instrumentation = shm_toc_lookup(toc, PARALLEL_KEY_INSTRUMENTATION, true);
+ instrumentation = shm_toc_lookup(toc, PARALLEL_KEY_NODE_INSTRUMENTATION, true);
if (instrumentation != NULL)
instrument_options = instrumentation->instrument_options;
jit_instrumentation = shm_toc_lookup(toc, PARALLEL_KEY_JIT_INSTRUMENTATION,
@@ -1545,11 +1534,12 @@ ParallelQueryMain(dsm_segment *seg, shm_toc *toc)
ExecutorFinish(queryDesc);
/* Report buffer/WAL usage during parallel execution. */
- buffer_usage = shm_toc_lookup(toc, PARALLEL_KEY_BUFFER_USAGE, false);
- wal_usage = shm_toc_lookup(toc, PARALLEL_KEY_WAL_USAGE, false);
- InstrEndParallelQuery(instr,
- &buffer_usage[ParallelWorkerNumber],
- &wal_usage[ParallelWorkerNumber]);
+ {
+ Instrumentation *worker_instr;
+
+ worker_instr = shm_toc_lookup(toc, PARALLEL_KEY_INSTRUMENTATION, false);
+ InstrEndParallelQuery(instr, &worker_instr[ParallelWorkerNumber]);
+ }
/* Report instrumentation data if any instrumentation options are set. */
if (instrumentation != NULL)
diff --git a/src/backend/executor/instrument.c b/src/backend/executor/instrument.c
index 2727e7b5ce..61297c5653 100644
--- a/src/backend/executor/instrument.c
+++ b/src/backend/executor/instrument.c
@@ -286,11 +286,11 @@ InstrStartParallelQuery(void)
/* report usage after parallel executor shutdown */
void
-InstrEndParallelQuery(QueryInstrumentation *qinstr, BufferUsage *bufusage, WalUsage *walusage)
+InstrEndParallelQuery(QueryInstrumentation *qinstr, Instrumentation *dst)
{
qinstr = InstrQueryStopFinalize(qinstr);
- memcpy(bufusage, &qinstr->instr.bufusage, sizeof(BufferUsage));
- memcpy(walusage, &qinstr->instr.walusage, sizeof(WalUsage));
+ memcpy(&dst->bufusage, &qinstr->instr.bufusage, sizeof(BufferUsage));
+ memcpy(&dst->walusage, &qinstr->instr.walusage, sizeof(WalUsage));
}
/*
@@ -306,12 +306,12 @@ InstrEndParallelQuery(QueryInstrumentation *qinstr, BufferUsage *bufusage, WalUs
* activity is accumulated.
*/
void
-InstrAccumParallelQuery(BufferUsage *bufusage, WalUsage *walusage)
+InstrAccumParallelQuery(Instrumentation *instr)
{
- BufferUsageAdd(&instr_stack.current->bufusage, bufusage);
- WalUsageAdd(&instr_stack.current->walusage, walusage);
+ BufferUsageAdd(&instr_stack.current->bufusage, &instr->bufusage);
+ WalUsageAdd(&instr_stack.current->walusage, &instr->walusage);
- WalUsageAdd(&pgWalUsage, walusage);
+ WalUsageAdd(&pgWalUsage, &instr->walusage);
}
/* Node instrumentation handling */
diff --git a/src/include/executor/execParallel.h b/src/include/executor/execParallel.h
index 5a2034811d..6c8b602d07 100644
--- a/src/include/executor/execParallel.h
+++ b/src/include/executor/execParallel.h
@@ -25,9 +25,8 @@ typedef struct ParallelExecutorInfo
{
PlanState *planstate; /* plan subtree we're running in parallel */
ParallelContext *pcxt; /* parallel context we're using */
- BufferUsage *buffer_usage; /* points to bufusage area in DSM */
- WalUsage *wal_usage; /* walusage area in DSM */
- SharedExecutorInstrumentation *instrumentation; /* optional */
+ Instrumentation *instrumentation; /* instrumentation area in DSM */
+ SharedExecutorInstrumentation *node_instrumentation; /* optional */
struct SharedJitInstrumentation *jit_instrumentation; /* optional */
dsa_area *area; /* points to DSA area in DSM */
dsa_pointer param_exec; /* serialized PARAM_EXEC parameters */
diff --git a/src/include/executor/instrument.h b/src/include/executor/instrument.h
index bce58acb11..e4fc9e7870 100644
--- a/src/include/executor/instrument.h
+++ b/src/include/executor/instrument.h
@@ -275,8 +275,8 @@ extern QueryInstrumentation *InstrQueryStopFinalize(QueryInstrumentation *instr)
extern void InstrQueryRememberNode(QueryInstrumentation *parent, NodeInstrumentation *instr);
pg_nodiscard extern QueryInstrumentation *InstrStartParallelQuery(void);
-extern void InstrEndParallelQuery(QueryInstrumentation *qinstr, BufferUsage *bufusage, WalUsage *walusage);
-extern void InstrAccumParallelQuery(BufferUsage *bufusage, WalUsage *walusage);
+extern void InstrEndParallelQuery(QueryInstrumentation *qinstr, Instrumentation *dst);
+extern void InstrAccumParallelQuery(Instrumentation *instr);
extern NodeInstrumentation *InstrAllocNode(int instrument_options,
bool async_mode);
--
2.43.0
[text/x-patch] nocfbot-0001-bufmgr-Return-whether-WaitReadBuffers-neede.patch (2.8K, ../../CAP53Pkx_x2WUtM40r6+GZMVCW5d2PE8KKYQc1=ArODn0-WYq+A@mail.gmail.com/3-nocfbot-0001-bufmgr-Return-whether-WaitReadBuffers-neede.patch)
download | inline diff:
From 6c03de986842a0707890dff3668b9d8273f44b0f Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH vnocfbot 1/3] bufmgr: Return whether WaitReadBuffers() needed
to wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 41a0baa344..15a11108b8 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1738,12 +1738,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1805,6 +1813,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1835,6 +1844,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1851,6 +1866,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4017896f95..a1ef04354d 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -249,7 +249,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.43.0
[text/x-patch] nocfbot-0003-instrumentation-Track-I-O-prefetch-info-and.patch (22.3K, ../../CAP53Pkx_x2WUtM40r6+GZMVCW5d2PE8KKYQc1=ArODn0-WYq+A@mail.gmail.com/4-nocfbot-0003-instrumentation-Track-I-O-prefetch-info-and.patch)
download | inline diff:
From 08821a5db5b7e78dd31f1911253d907a391c6b69 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Sun, 15 Mar 2026 23:29:15 -0700
Subject: [PATCH vnocfbot 3/3] instrumentation: Track I/O prefetch info and
show with EXPLAIN (IO)
This adds details about AIO / prefetch for executor nodes using the
ReadStream API, currently SeqScan and BitmapHeapScan, into the new
IOUsage information added to Instrumentation. This can be viewed
through the new EXPLAIN (IO) information, or could be tracked by
other interested callers through the stack-based instrumentation
mechanism.
The ReadStream tracks the statistics unconditionally, i.e. even
outside EXPLAIN ANALYZE etc. The amount of statistics is trivial
(a handful of integer counters), it's not worth gating this by a
flag.
Author: Lukas Fittl <lukas@fittl.com>
Author: Tomas Vondra <tomas@vondra.me>
Reviewed By:
Discussion: https://www.postgresql.org/message-id/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/backend/commands/explain.c | 75 +++++++++++++++++++++++++--
src/backend/commands/explain_state.c | 8 +++
src/backend/executor/execMain.c | 6 +--
src/backend/executor/instrument.c | 54 ++++++++++++++-----
src/backend/storage/aio/read_stream.c | 33 +++++++++++-
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument.h | 31 +++++++++++
src/tools/pgindent/typedefs.list | 1 +
8 files changed, 189 insertions(+), 20 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8a641f9d05..6eca0e2051 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -145,6 +145,7 @@ static const char *explain_get_index_name(Oid indexId);
static bool peek_buffer_usage(ExplainState *es, const BufferUsage *usage);
static void show_buffer_usage(ExplainState *es, const BufferUsage *usage, const char *title);
static void show_wal_usage(ExplainState *es, const WalUsage *usage);
+static void show_io_usage(ExplainState *es, const IOUsage *usage);
static void show_memory_counters(ExplainState *es,
const MemoryContextCounters *mem_counters);
static void show_result_replacement_info(Result *result, ExplainState *es);
@@ -509,6 +510,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
instrument_option |= INSTRUMENT_BUFFERS;
if (es->wal)
instrument_option |= INSTRUMENT_WAL;
+ if (es->io)
+ instrument_option |= INSTRUMENT_IO;
/*
* We always collect timing for the entire statement, even when node-level
@@ -2281,14 +2284,16 @@ ExplainNode(PlanState *planstate, List *ancestors,
}
}
- /* Show buffer/WAL usage */
+ /* Show buffer/WAL/IO usage */
if (es->buffers && planstate->instrument)
show_buffer_usage(es, &planstate->instrument->instr.bufusage, NULL);
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->instr.walusage);
+ if (es->io && planstate->instrument)
+ show_io_usage(es, &planstate->instrument->instr.iousage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerNodeInstrumentation *w = planstate->worker_instrument;
@@ -2305,6 +2310,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->instr.bufusage, NULL);
if (es->wal)
show_wal_usage(es, &instrument->instr.walusage);
+ if (es->io)
+ show_io_usage(es, &instrument->instr.iousage);
ExplainCloseWorker(n, es);
}
}
@@ -4343,6 +4350,68 @@ show_wal_usage(ExplainState *es, const WalUsage *usage)
}
}
+/*
+ * Show I/O prefetch usage details.
+ */
+static void
+show_io_usage(ExplainState *es, const IOUsage *usage)
+{
+ /* Nothing to show if no buffers were returned */
+ if (usage->count <= 0)
+ return;
+
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%" PRId64 " capacity=%" PRId64,
+ (usage->distance_sum * 1.0 / usage->count),
+ usage->distance_max,
+ usage->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (usage->stall_count > 0 || usage->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRId64,
+ usage->stall_count);
+
+ if (usage->io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (usage->io_blocks * 1.0 / usage->io_count),
+ (usage->ios_in_progress * 1.0 / usage->io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (usage->distance_sum * 1.0 / usage->count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ usage->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ usage->distance_capacity, es);
+ ExplainPropertyInteger("Stalls", NULL,
+ usage->stall_count, es);
+
+ if (usage->io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (usage->io_blocks * 1.0 / usage->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (usage->ios_in_progress * 1.0 / usage->io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+}
+
/*
* Show memory usage details.
*/
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e50..b5129f4914 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -115,6 +115,8 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
}
else if (strcmp(opt->defname, "memory") == 0)
es->memory = defGetBoolean(opt);
+ else if (strcmp(opt->defname, "io") == 0)
+ es->io = defGetBoolean(opt);
else if (strcmp(opt->defname, "serialize") == 0)
{
if (opt->arg)
@@ -185,6 +187,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 82253317e9..de0795c944 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -334,7 +334,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
* Start up required top-level instrumentation stack for WAL/buffer
* tracking
*/
- if (!queryDesc->totaltime && (estate->es_instrument & (INSTRUMENT_BUFFERS | INSTRUMENT_WAL)))
+ if (!queryDesc->totaltime && (estate->es_instrument & (INSTRUMENT_BUFFERS | INSTRUMENT_WAL | INSTRUMENT_IO)))
queryDesc->totaltime = InstrQueryAlloc(estate->es_instrument);
if (queryDesc->totaltime)
@@ -347,7 +347,7 @@ standard_ExecutorRun(QueryDesc *queryDesc,
* after the first call to InstrQueryStart has pushed the parent
* entry.
*/
- if ((estate->es_instrument & (INSTRUMENT_BUFFERS | INSTRUMENT_WAL)) &&
+ if ((estate->es_instrument & (INSTRUMENT_BUFFERS | INSTRUMENT_WAL | INSTRUMENT_IO)) &&
!queryDesc->already_executed)
ExecRememberNodeInstrumentation(queryDesc->planstate,
queryDesc->totaltime);
@@ -1535,7 +1535,7 @@ ExecFinalizeTriggerInstrumentation(EState *estate)
{
TriggerInstrumentation *ti = rInfo->ri_TrigInstrument;
- if (ti && (ti->instr.need_bufusage || ti->instr.need_walusage))
+ if (ti && (ti->instr.need_bufusage || ti->instr.need_walusage || ti->instr.need_iousage))
InstrAccum(instr_stack.current, &ti->instr);
}
}
diff --git a/src/backend/executor/instrument.c b/src/backend/executor/instrument.c
index 61297c5653..c8f182e7c5 100644
--- a/src/backend/executor/instrument.c
+++ b/src/backend/executor/instrument.c
@@ -46,6 +46,7 @@ InstrInitOptions(Instrumentation *instr, int instrument_options)
{
instr->need_bufusage = (instrument_options & INSTRUMENT_BUFFERS) != 0;
instr->need_walusage = (instrument_options & INSTRUMENT_WAL) != 0;
+ instr->need_iousage = (instrument_options & INSTRUMENT_IO) != 0;
instr->need_timer = (instrument_options & INSTRUMENT_TIMER) != 0;
}
@@ -76,7 +77,7 @@ InstrStart(Instrumentation *instr)
if (instr->need_timer)
InstrStartTimer(instr);
- if (instr->need_bufusage || instr->need_walusage)
+ if (instr->need_bufusage || instr->need_walusage || instr->need_iousage)
InstrPushStack(instr);
}
@@ -86,7 +87,7 @@ InstrStop(Instrumentation *instr)
if (instr->need_timer)
InstrStopTimer(instr);
- if (instr->need_bufusage || instr->need_walusage)
+ if (instr->need_bufusage || instr->need_walusage || instr->need_iousage)
InstrPopStack(instr);
}
@@ -197,7 +198,9 @@ InstrQueryAlloc(int instrument_options)
* survives transaction abort — ResourceOwner release needs to access
* it.
*/
- if ((instrument_options & INSTRUMENT_BUFFERS) != 0 || (instrument_options & INSTRUMENT_WAL) != 0)
+ if ((instrument_options & INSTRUMENT_BUFFERS) != 0 ||
+ (instrument_options & INSTRUMENT_WAL) != 0 ||
+ (instrument_options & INSTRUMENT_IO) != 0)
instr = MemoryContextAllocZero(TopMemoryContext, sizeof(QueryInstrumentation));
else
instr = palloc0(sizeof(QueryInstrumentation));
@@ -213,7 +216,7 @@ InstrQueryStart(QueryInstrumentation *qinstr)
{
InstrStart(&qinstr->instr);
- if (qinstr->instr.need_bufusage || qinstr->instr.need_walusage)
+ if (qinstr->instr.need_bufusage || qinstr->instr.need_walusage || qinstr->instr.need_iousage)
{
Assert(CurrentResourceOwner != NULL);
qinstr->owner = CurrentResourceOwner;
@@ -228,7 +231,7 @@ InstrQueryStop(QueryInstrumentation *qinstr)
{
InstrStop(&qinstr->instr);
- if (qinstr->instr.need_bufusage || qinstr->instr.need_walusage)
+ if (qinstr->instr.need_bufusage || qinstr->instr.need_walusage || qinstr->instr.need_iousage)
{
Assert(qinstr->owner != NULL);
ResourceOwnerForgetInstrumentation(qinstr->owner, qinstr);
@@ -243,7 +246,7 @@ InstrQueryStopFinalize(QueryInstrumentation *qinstr)
InstrStopFinalize(&qinstr->instr);
- if (!qinstr->instr.need_bufusage && !qinstr->instr.need_walusage)
+ if (!qinstr->instr.need_bufusage && !qinstr->instr.need_walusage && !qinstr->instr.need_iousage)
return qinstr;
Assert(qinstr->owner != NULL);
@@ -270,7 +273,7 @@ InstrQueryStopFinalize(QueryInstrumentation *qinstr)
void
InstrQueryRememberNode(QueryInstrumentation *parent, NodeInstrumentation *child)
{
- if (child->instr.need_bufusage || child->instr.need_walusage)
+ if (child->instr.need_bufusage || child->instr.need_walusage || child->instr.need_iousage)
dlist_push_head(&parent->unfinalized_children, &child->unfinalized_node);
}
@@ -278,7 +281,7 @@ InstrQueryRememberNode(QueryInstrumentation *parent, NodeInstrumentation *child)
QueryInstrumentation *
InstrStartParallelQuery(void)
{
- QueryInstrumentation *qinstr = InstrQueryAlloc(INSTRUMENT_BUFFERS | INSTRUMENT_WAL);
+ QueryInstrumentation *qinstr = InstrQueryAlloc(INSTRUMENT_BUFFERS | INSTRUMENT_WAL | INSTRUMENT_IO);
InstrQueryStart(qinstr);
return qinstr;
@@ -291,6 +294,7 @@ InstrEndParallelQuery(QueryInstrumentation *qinstr, Instrumentation *dst)
qinstr = InstrQueryStopFinalize(qinstr);
memcpy(&dst->bufusage, &qinstr->instr.bufusage, sizeof(BufferUsage));
memcpy(&dst->walusage, &qinstr->instr.walusage, sizeof(WalUsage));
+ memcpy(&dst->iousage, &qinstr->instr.iousage, sizeof(IOUsage));
}
/*
@@ -310,6 +314,7 @@ InstrAccumParallelQuery(Instrumentation *instr)
{
BufferUsageAdd(&instr_stack.current->bufusage, &instr->bufusage);
WalUsageAdd(&instr_stack.current->walusage, &instr->walusage);
+ IOUsageAdd(&instr_stack.current->iousage, &instr->iousage);
WalUsageAdd(&pgWalUsage, &instr->walusage);
}
@@ -329,7 +334,9 @@ InstrAllocNode(int instrument_options, bool async_mode)
* utility commands that restart transactions, which would require a
* context that survives longer (EXPLAIN ANALYZE is fine).
*/
- if ((instrument_options & INSTRUMENT_BUFFERS) != 0 || (instrument_options & INSTRUMENT_WAL) != 0)
+ if ((instrument_options & INSTRUMENT_BUFFERS) != 0 ||
+ (instrument_options & INSTRUMENT_WAL) != 0 ||
+ (instrument_options & INSTRUMENT_IO) != 0)
instr = MemoryContextAlloc(TopTransactionContext, sizeof(NodeInstrumentation));
else
instr = palloc(sizeof(NodeInstrumentation));
@@ -392,7 +399,7 @@ InstrStopNode(NodeInstrumentation *instr, double nTuples)
InstrStopNodeTimer(instr);
/* Only pop the stack, accumulation runs in InstrFinalizeNode */
- if (instr->instr.need_bufusage || instr->instr.need_walusage)
+ if (instr->instr.need_bufusage || instr->instr.need_walusage || instr->instr.need_iousage)
InstrPopStack(&instr->instr);
instr->running = true;
@@ -407,7 +414,7 @@ InstrFinalizeNode(NodeInstrumentation *instr, Instrumentation *parent)
NodeInstrumentation *dst;
/* If we didn't use stack based instrumentation, nothing to be done */
- if (!instr->instr.need_bufusage && !instr->instr.need_walusage)
+ if (!instr->instr.need_bufusage && !instr->instr.need_walusage && !instr->instr.need_iousage)
return instr;
/* Copy into per-query memory context */
@@ -418,7 +425,7 @@ InstrFinalizeNode(NodeInstrumentation *instr, Instrumentation *parent)
InstrAccum(parent, &dst->instr);
/* Unregister from query's unfinalized list before freeing */
- if (instr->instr.need_bufusage || instr->instr.need_walusage)
+ if (instr->instr.need_bufusage || instr->instr.need_walusage || instr->instr.need_iousage)
dlist_delete(&instr->unfinalized_node);
pfree(instr);
@@ -489,6 +496,9 @@ InstrAggNode(NodeInstrumentation *dst, NodeInstrumentation *add)
if (dst->instr.need_walusage)
WalUsageAdd(&dst->instr.walusage, &add->instr.walusage);
+
+ if (dst->instr.need_iousage)
+ IOUsageAdd(&dst->instr.iousage, &add->instr.iousage);
}
/*
@@ -598,7 +608,8 @@ InstrNodeSetupExecProcNode(NodeInstrumentation *instr)
{
bool need_timer = instr->instr.need_timer;
bool need_buf = (instr->instr.need_bufusage ||
- instr->instr.need_walusage);
+ instr->instr.need_walusage ||
+ instr->instr.need_iousage);
if (need_timer && need_buf)
return ExecProcNodeInstrFull;
@@ -649,6 +660,7 @@ InstrAccum(Instrumentation *dst, Instrumentation *add)
BufferUsageAdd(&dst->bufusage, &add->bufusage);
WalUsageAdd(&dst->walusage, &add->walusage);
+ IOUsageAdd(&dst->iousage, &add->iousage);
}
/* dst += add */
@@ -684,6 +696,22 @@ WalUsageAdd(WalUsage *dst, const WalUsage *add)
dst->wal_buffers_full += add->wal_buffers_full;
}
+/* dst += add (using max semantics for distance_max and distance_capacity) */
+void
+IOUsageAdd(IOUsage *dst, const IOUsage *add)
+{
+ dst->count += add->count;
+ dst->distance_sum += add->distance_sum;
+ if (add->distance_max > dst->distance_max)
+ dst->distance_max = add->distance_max;
+ if (add->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = add->distance_capacity;
+ dst->stall_count += add->stall_count;
+ dst->io_count += add->io_count;
+ dst->io_blocks += add->io_blocks;
+ dst->ios_in_progress += add->ios_in_progress;
+}
+
void
WalUsageAccumDiff(WalUsage *dst, const WalUsage *add, const WalUsage *sub)
{
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74a..6d2285beb1 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -71,6 +71,7 @@
*/
#include "postgres.h"
+#include "executor/instrument.h"
#include "miscadmin.h"
#include "storage/aio.h"
#include "storage/fd.h"
@@ -172,6 +173,19 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update IO instrumentation when returning a buffer to the consumer.
+ * Records the current look-ahead depth for averaging.
+ */
+static inline void
+read_stream_instr_update(ReadStream *stream)
+{
+ INSTR_IOUSAGE_INCR(count);
+ INSTR_IOUSAGE_ADD(distance_sum, stream->pinned_buffers);
+ INSTR_IOUSAGE_MAX(distance_max, stream->pinned_buffers);
+ INSTR_IOUSAGE_MAX(distance_capacity, stream->max_pinned_buffers);
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +394,11 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* Update I/O stats */
+ INSTR_IOUSAGE_INCR(io_count);
+ INSTR_IOUSAGE_ADD(io_blocks, nblocks);
+ INSTR_IOUSAGE_ADD(ios_in_progress, stream->ios_in_progress);
}
/*
@@ -851,6 +870,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_instr_update(stream);
return buffer;
}
@@ -860,6 +880,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* Since we executed IO synchronously, count it as a stall */
+ INSTR_IOUSAGE_INCR(stall_count);
}
else
{
@@ -871,6 +894,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
stream->fast_path = false;
+ read_stream_instr_update(stream);
return buffer;
}
#endif
@@ -916,12 +940,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we needed to wait for I/O */
+ if (needed_wait)
+ INSTR_IOUSAGE_INCR(stall_count);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1010,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_instr_update(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 0b695f7d81..801e422437 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -54,6 +54,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument.h b/src/include/executor/instrument.h
index e4fc9e7870..0b7a84fc55 100644
--- a/src/include/executor/instrument.h
+++ b/src/include/executor/instrument.h
@@ -58,6 +58,22 @@ typedef struct WalUsage
int64 wal_buffers_full; /* # of times the WAL buffers became full */
} WalUsage;
+/*
+ * IOUsage tracks I/O prefetch activity that can be measured per executor
+ * node and displayed by EXPLAIN (ANALYZE, IO).
+ */
+typedef struct IOUsage
+{
+ int64 count; /* # of buffers returned */
+ int64 distance_sum; /* sum of look-ahead distances */
+ int64 distance_max; /* max look-ahead distance observed */
+ int64 distance_capacity; /* max possible look-ahead */
+ int64 stall_count; /* # of I/O stalls */
+ int64 io_count; /* # of I/O operations */
+ int64 io_blocks; /* total blocks across I/Os */
+ int64 ios_in_progress; /* sum of in-progress I/Os */
+} IOUsage;
+
/* Flag bits included in InstrAlloc's instrument_options bitmask */
typedef enum InstrumentOption
{
@@ -65,6 +81,7 @@ typedef enum InstrumentOption
INSTRUMENT_BUFFERS = 1 << 1, /* needs buffer usage */
INSTRUMENT_ROWS = 1 << 2, /* needs row count */
INSTRUMENT_WAL = 1 << 3, /* needs WAL usage */
+ INSTRUMENT_IO = 1 << 4, /* needs I/O prefetch usage */
INSTRUMENT_ALL = PG_INT32_MAX
} InstrumentOption;
@@ -92,12 +109,14 @@ typedef struct Instrumentation
bool need_timer; /* true if we need timer data */
bool need_bufusage; /* true if we need buffer usage data */
bool need_walusage; /* true if we need WAL usage data */
+ bool need_iousage; /* true if we need I/O prefetch data */
/* Internal state keeping: */
instr_time starttime; /* start time of last InstrStart */
/* Accumulated statistics: */
instr_time total; /* total runtime */
BufferUsage bufusage; /* total buffer usage */
WalUsage walusage; /* total WAL usage */
+ IOUsage iousage; /* total I/O prefetch usage */
} Instrumentation;
/*
@@ -301,6 +320,7 @@ extern void BufferUsageAdd(BufferUsage *dst, const BufferUsage *add);
extern void WalUsageAdd(WalUsage *dst, const WalUsage *add);
extern void WalUsageAccumDiff(WalUsage *dst, const WalUsage *add,
const WalUsage *sub);
+extern void IOUsageAdd(IOUsage *dst, const IOUsage *add);
#define INSTR_BUFUSAGE_INCR(fld) do { \
instr_stack.current->bufusage.fld++; \
@@ -324,4 +344,15 @@ extern void WalUsageAccumDiff(WalUsage *dst, const WalUsage *add,
instr_stack.current->walusage.fld += val; \
} while(0)
+#define INSTR_IOUSAGE_INCR(fld) do { \
+ instr_stack.current->iousage.fld++; \
+ } while(0)
+#define INSTR_IOUSAGE_ADD(fld,val) do { \
+ instr_stack.current->iousage.fld += val; \
+ } while(0)
+#define INSTR_IOUSAGE_MAX(fld,val) do { \
+ if ((val) > instr_stack.current->iousage.fld) \
+ instr_stack.current->iousage.fld = (val); \
+ } while(0)
+
#endif /* INSTRUMENT_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1c9be944c5..985662a252 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1336,6 +1336,7 @@ InvalMessageArray
InvalidationInfo
InvalidationMsgsGroup
IoMethodOps
+IOUsage
IpcMemoryId
IpcMemoryKey
IpcMemoryState
--
2.43.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 07:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
@ 2026-03-16 17:29 ` Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 18:26 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-03-16 17:29 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Tomas Vondra <tomas@vondra.me>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Andres Freund <andres@anarazel.de>
On Mon, Mar 16, 2026 at 3:08 AM Lukas Fittl <lukas@fittl.com> wrote:
>
> I'm 50/50 if hiding this behind a new option really makes sense - if
> its cheap enough to always capture, why not always show it?
>
> e.g. we could consider doing with this what we did with BUFFERS
> recently, which is to enable it by default. If someone finds that too
> visually busy, they could still do IO OFF.
I like this idea.
> This also does make me wonder a bit what we should do about I/O
> timings. Conceptually they'd belong closer to IO now than BUFFERS..
I could see moving the timings to the I/O line.
> > The second line "I/O" is about the I/O requests actually issued - how
> > many times we had to wait for the block (when we get to process it),
> > average size of a request (in BLCKSZ blocks), and average number of
> > in-progress requests.
>
> I wonder if we could somehow consolidate this into one line for the
> text format? (specifically, moving prefetch into "I/O" at the end?)
Next release, I'm hopeful we'll get write combining in and then want
to include write IO details here. Not a reason to avoid making it one
line now, though. However, I think it will be a very long line...
> I'm also not sure if "max" is really that useful, vs capacity?
I find it very helpful. If you keep increasing
effective_io_concurrency and io_combine_limit and don't see the max
increasing, I think it is helpful to know what the configured possible
limit would be vs what the read stream actually got you up to. And
because you can set effective_io_concurrency and io_combine_limit per
query, it is nice to know what this value was at the time the query
was run.
> I feel like something is off about the complexity of having each node
> type ferry back the information. e.g. when you're implementing the
> support for index prefetching, it'll require a bunch more changes. In
> my mind, there is a reason we have a related problem that we solved
> with the current pgBufferUsage, instead of dealing with that on a
> per-node basis. I really feel we should have a more generic way of
> dealing with this.
<--snip-->
> I've attached a prototype of how that could look like (apply the other
> patch set first, v8, see commit fest entry [1] - also attached a
> preparatory refactoring of using "Instrumentation" for parallel query
> reporting, which avoids having individual structs there).
The patch footprint is _much_ nicer with your stack-based
instrumentation. Very cool. I'll leave it to Tomas whether he wants to
create a dependency on your big project a few weeks before feature
freeze, though.
> Its also worth noting that this would make it trivial to output this
> information for utility commands that have read stream support, or
> show aggregate statistics in pg_stat_statements/etc.
Yes this would be a major bonus. Even for currently possible users,
this hasn't been extended to TID Range Scan -- which involves more LOC
and boilerplate.
- Melanie
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 07:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-16 17:29 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-03-16 18:26 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 00:15 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-16 18:26 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; Lukas Fittl <lukas@fittl.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Andres Freund <andres@anarazel.de>
On 3/16/26 18:29, Melanie Plageman wrote:
> On Mon, Mar 16, 2026 at 3:08 AM Lukas Fittl <lukas@fittl.com> wrote:
>>
>> I'm 50/50 if hiding this behind a new option really makes sense - if
>> its cheap enough to always capture, why not always show it?
>>
>> e.g. we could consider doing with this what we did with BUFFERS
>> recently, which is to enable it by default. If someone finds that too
>> visually busy, they could still do IO OFF.
>
> I like this idea.
>
Maybe. I think it's important not to overwhelm the users with too much
information, and so my intuition was to keep it OFF by default. But if
the agreement is that it should be opt-out, I can live with that.
>> This also does make me wonder a bit what we should do about I/O
>> timings. Conceptually they'd belong closer to IO now than BUFFERS..
>
> I could see moving the timings to the I/O line.
>
Moving the I/O timings to the "IO" line seems reasonable to me.
>>> The second line "I/O" is about the I/O requests actually issued - how
>>> many times we had to wait for the block (when we get to process it),
>>> average size of a request (in BLCKSZ blocks), and average number of
>>> in-progress requests.
>>
>> I wonder if we could somehow consolidate this into one line for the
>> text format? (specifically, moving prefetch into "I/O" at the end?)
>
> Next release, I'm hopeful we'll get write combining in and then want
> to include write IO details here. Not a reason to avoid making it one
> line now, though. However, I think it will be a very long line...
>
Let's not make the lines too long. The distance information is not
really about I/O, it's about an internal queue / heuristics.
FWIW I assume we'd want to track some of the metrics (average IO size,
number of IOs, ...) separate for reads and writes, so I'm not sure those
will go to the same line too.
>> I'm also not sure if "max" is really that useful, vs capacity?
>
> I find it very helpful. If you keep increasing
> effective_io_concurrency and io_combine_limit and don't see the max
> increasing, I think it is helpful to know what the configured possible
> limit would be vs what the read stream actually got you up to. And
> because you can set effective_io_concurrency and io_combine_limit per
> query, it is nice to know what this value was at the time the query
> was run.
>
I'm not sure. Shouldn't the average distance give us about the same
thing? Sure, it'll never be exactly the maximum, but if the prefetch
behavior is reasonably stable, it should not be too far.
It'll be different if the prefetch distance varies a lot (e.g. because
parts of the table have vastly different cache hit ratios). But then
it's hard to interpret the "max" too, I think.
So maybe just average + capacity might suffice?
>> I feel like something is off about the complexity of having each node
>> type ferry back the information. e.g. when you're implementing the
>> support for index prefetching, it'll require a bunch more changes. In
>> my mind, there is a reason we have a related problem that we solved
>> with the current pgBufferUsage, instead of dealing with that on a
>> per-node basis. I really feel we should have a more generic way of
>> dealing with this.
> <--snip-->
>> I've attached a prototype of how that could look like (apply the other
>> patch set first, v8, see commit fest entry [1] - also attached a
>> preparatory refactoring of using "Instrumentation" for parallel query
>> reporting, which avoids having individual structs there).
>
> The patch footprint is _much_ nicer with your stack-based
> instrumentation. Very cool. I'll leave it to Tomas whether he wants to
> create a dependency on your big project a few weeks before feature
> freeze, though.
>
I don't know. I have not looked at the stack-based instrumentation yet,
so I have no idea how complex or how close to committable it is.
>> Its also worth noting that this would make it trivial to output this
>> information for utility commands that have read stream support, or
>> show aggregate statistics in pg_stat_statements/etc.
>
> Yes this would be a major bonus. Even for currently possible users,
> this hasn't been extended to TID Range Scan -- which involves more LOC
> and boilerplate.
>
Oh, I forgot about the TID scan ...
I agree doing this in a way that does not require changes to TAM
interface would be nice.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 07:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-16 17:29 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 18:26 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-17 00:15 ` Tomas Vondra <tomas@vondra.me>
0 siblings, 0 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-03-17 00:15 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; Lukas Fittl <lukas@fittl.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Andres Freund <andres@anarazel.de>
On 3/16/26 19:26, Tomas Vondra wrote:
> On 3/16/26 18:29, Melanie Plageman wrote:
>> On Mon, Mar 16, 2026 at 3:08 AM Lukas Fittl <lukas@fittl.com> wrote:
> ...
>
>>> I feel like something is off about the complexity of having each node
>>> type ferry back the information. e.g. when you're implementing the
>>> support for index prefetching, it'll require a bunch more changes. In
>>> my mind, there is a reason we have a related problem that we solved
>>> with the current pgBufferUsage, instead of dealing with that on a
>>> per-node basis. I really feel we should have a more generic way of
>>> dealing with this.
>> <--snip-->
>>> I've attached a prototype of how that could look like (apply the other
>>> patch set first, v8, see commit fest entry [1] - also attached a
>>> preparatory refactoring of using "Instrumentation" for parallel query
>>> reporting, which avoids having individual structs there).
>>
It looks reasonable, and it's nice to not have to worry about passing
the data through the TAM interface etc. Having to do stuff like this:
- if (instr->instr.need_bufusage || instr->instr.need_walusage)
+ if (instr->instr.need_bufusage || instr->instr.need_walusage ||
instr->instr.need_iousage)
in so many places is not great. It'd really benefit from some macro that
says "if any of the pieces is needed ...".
>> The patch footprint is _much_ nicer with your stack-based
>> instrumentation. Very cool. I'll leave it to Tomas whether he wants to
>> create a dependency on your big project a few weeks before feature
>> freeze, though.
>>
>
> I don't know. I have not looked at the stack-based instrumentation yet,
> so I have no idea how complex or how close to committable it is.
>
I've started looking at the stack tracking patch only today, and it
looks in pretty good shape. I have no clear idea how close to being
committed it is, so I'm hesitant to make this patch depend on it.
AFAICS the stack-based tracking is meant for resources with just one
instance. E.g. there's only one WAL, or one buffer pool, so when a node
says "update WAL usage" or "increment buffer hit/read", there's one
counter to update. But will that necessarily apply for read streams?
AFAIK existing scans use only a single read stream, but that's only
because none of the built-in index AMs uses a read stream. If we get
index prefetching in PG19, or if any index AM (e.g. pgvector) chooses to
use a read stream internally, how will that work? Will we "merge" the
stats from all read streams used by the node, or what will happen?
I realize this is a similar issue the 0007 part of the stack-based
tracing patch deals with - tracking buffers for index/table separately.
And it does that by making nodeIndexscan.c quite a bit more complex by
expanding index_getnext_slot(). Doesn't seem great, but not sure. I
suppose nodeIndexonlyscan.c will have to do something similar.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-16 17:14 ` Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-03-16 17:14 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Sun, Mar 15, 2026 at 3:49 PM Tomas Vondra <tomas@vondra.me> wrote:
>
> - If the separation between TAM and the low-level instrumentation clear
> enough? Or is the ReadStreamInstrumentation "leaking" somewhere? For
> example, is it OK it's in SeqScanInstrumentation?
Personally, I don't like having both structs
(ReadStreamInstrumentation and TableScanStatsData).
The executor nodes (SeqScanState, BitmapHeapScanState) already embed
ReadStreamInstrumentation directly in their instrumentation structs,
so we already have a reference to the read stream in table AM-agnostic
code. Having a second identical struct means maintaining two
definitions without any actual benefit.
> But I'm sure there are other questions I haven't thought of.
I see a couple more issues with the counting in read_stream.c.
You are double-counting stalls for synchronous IO. You increment
stalls in read_stream_next_buffer() but we actually execute
synchronous IO in WaitReadBuffers and return needed_wait as true,
which will count a stall again.
You are not counting fast path IOs because those don't go through
read_stream_start_pending_read() and instead are started directly by
StartReadBuffer() in read_stream_next_buffer(). Simple diff attached
should fix this.
Also, per worker stats are not displayed when BUFFERS false is passed
even with IO true because of a small oversight. I fixed it in the
attached diff.
- Melanie
Attachments:
[text/x-patch] nocfbot-0001-fixups.patch (1.5K, ../../CAAKRu_byaXSbanJE_q0P2ngd46hQBu9jeBde72NOaKy=N83XnQ@mail.gmail.com/2-nocfbot-0001-fixups.patch)
download | inline diff:
From 176578c63fc2833157646f668bd0a007253fc5aa Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 16 Mar 2026 13:10:46 -0400
Subject: [PATCH] fixups
---
src/backend/commands/explain.c | 2 +-
src/backend/storage/aio/read_stream.c | 4 +---
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 41c9cabd9ae..5491b56d12b 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2306,7 +2306,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_wal_usage(es, &planstate->instrument->walusage);
/* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 627afafef05..1227721ab5b 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -904,9 +904,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
-
- /* since we executed IO synchronously, count it as a stall */
- stream->stats.stall_count += 1;
+ read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
}
else
{
--
2.43.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-03-16 17:49 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-16 17:49 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/16/26 18:14, Melanie Plageman wrote:
> On Sun, Mar 15, 2026 at 3:49 PM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> - If the separation between TAM and the low-level instrumentation clear
>> enough? Or is the ReadStreamInstrumentation "leaking" somewhere? For
>> example, is it OK it's in SeqScanInstrumentation?
>
> Personally, I don't like having both structs
> (ReadStreamInstrumentation and TableScanStatsData).
>
> The executor nodes (SeqScanState, BitmapHeapScanState) already embed
> ReadStreamInstrumentation directly in their instrumentation structs,
> so we already have a reference to the read stream in table AM-agnostic
> code. Having a second identical struct means maintaining two
> definitions without any actual benefit.
>
That's a good point. The executor nodes should not embed anything
readstream-specific, not even in the instrumentation part. AFAICS the
cleanest way would be to replace this with the TableScanStatsData.
However, ReadStreamInstrumentation is not really ReadStream specific,
except for the name. I'm actually wondering if we should have just one
struct, that'd be enough.
>> But I'm sure there are other questions I haven't thought of.
>
> I see a couple more issues with the counting in read_stream.c.
>
> You are double-counting stalls for synchronous IO. You increment
> stalls in read_stream_next_buffer() but we actually execute
> synchronous IO in WaitReadBuffers and return needed_wait as true,
> which will count a stall again.
>
> You are not counting fast path IOs because those don't go through
> read_stream_start_pending_read() and instead are started directly by
> StartReadBuffer() in read_stream_next_buffer(). Simple diff attached
> should fix this.
>
> Also, per worker stats are not displayed when BUFFERS false is passed
> even with IO true because of a small oversight. I fixed it in the
> attached diff.
>
Thanks!
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-17 17:51 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-17 17:51 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/16/26 18:49, Tomas Vondra wrote:
> On 3/16/26 18:14, Melanie Plageman wrote:
>> ...
>>
>> I see a couple more issues with the counting in read_stream.c.
>>
>> You are double-counting stalls for synchronous IO. You increment
>> stalls in read_stream_next_buffer() but we actually execute
>> synchronous IO in WaitReadBuffers and return needed_wait as true,
>> which will count a stall again.
>>
>> You are not counting fast path IOs because those don't go through
>> read_stream_start_pending_read() and instead are started directly by
>> StartReadBuffer() in read_stream_next_buffer(). Simple diff attached
>> should fix this.
>>
>> Also, per worker stats are not displayed when BUFFERS false is passed
>> even with IO true because of a small oversight. I fixed it in the
>> attached diff.
>>
>
> Thanks!
>
Here's a v2 with your fixes merged. No other changes.
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v2-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch (2.8K, ../../0fa0794e-7e34-4fdb-8e35-a59ef8ce5246@vondra.me/2-v2-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch)
download | inline diff:
From 0136bf1c9d5ba2563c306fb9ab5a8c7b8a0b0912 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH v2 1/2] bufmgr: Return whether WaitReadBuffers() needed to
wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 00bc609529a..c8ec7fb4f53 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1738,12 +1738,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1805,6 +1813,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1835,6 +1844,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1851,6 +1866,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4017896f951..a1ef04354dd 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -249,7 +249,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.53.0
[text/x-patch] v2-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE-VE.patch (35.1K, ../../0fa0794e-7e34-4fdb-8e35-a59ef8ce5246@vondra.me/3-v2-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE-VE.patch)
download | inline diff:
From 874af056687bb4a21f4b135c8fcb88a28de070b9 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Thu, 12 Mar 2026 02:04:07 +0100
Subject: [PATCH v2 2/2] explain: show prefetch stats in EXPLAIN (ANALYZE,
VERBOSE)
This adds details about AIO / prefetch for a number of executor nodes
using the ReadStream, notably:
- SeqScan
- BitmapHeapScan
The statistics is tracked by the ReadStream, and then propagated up
through the table AM interface.
The ReadStream tracks the statistics unconditionally, i.e. even outside
EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
integer counters), it's not worth gating this by a flag.
The TAM gets one new callback "scan_stats", to collect stats from the
scan (which fetch tuples from the TAM). There is also a new struct
TableScanStatsData/TableScanStats to separate the statistics from the
actual TAM implementation.
This required improving SeqScan to properly setup shared instrumentation
between the leader and workers, collect it at the end of a scan, etc.
---
doc/src/sgml/ref/explain.sgml | 12 +
src/backend/access/heap/heapam.c | 32 +++
src/backend/access/heap/heapam_handler.c | 1 +
src/backend/commands/explain.c | 266 +++++++++++++++++++++-
src/backend/commands/explain_state.c | 10 +
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeBitmapHeapscan.c | 16 ++
src/backend/executor/nodeSeqscan.c | 123 +++++++++-
src/backend/storage/aio/read_stream.c | 64 +++++-
src/include/access/heapam.h | 1 +
src/include/access/relscan.h | 27 +++
src/include/access/tableam.h | 17 ++
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 45 +++-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 2 +
src/include/storage/read_stream.h | 3 +
17 files changed, 617 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 7dee77fd366..9da16c77d73 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -295,6 +296,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>FALSE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e5bd062de77..b13f843cfad 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1416,6 +1416,38 @@ heap_endscan(TableScanDesc sscan)
pfree(scan);
}
+/*
+ * heap_scan_stats
+ * return stats collected by the read stream
+ *
+ * Returns NULL if the scan is not using a read stream.
+ */
+TableScanStats
+heap_scan_stats(TableScanDesc sscan)
+{
+ HeapScanDesc scan = (HeapScanDesc) sscan;
+ ReadStreamInstrumentation stats;
+ TableScanStats res;
+
+ if (!scan->rs_read_stream)
+ return NULL;
+
+ stats = read_stream_prefetch_stats(scan->rs_read_stream);
+
+ res = palloc0(sizeof(TableScanStatsData));
+
+ res->prefetch_count = stats.prefetch_count;
+ res->distance_sum = stats.distance_sum;
+ res->distance_max = stats.distance_max;
+ res->distance_capacity = stats.distance_capacity;
+ res->stall_count = stats.stall_count;
+ res->io_count = stats.io_count;
+ res->io_nblocks = stats.io_nblocks;
+ res->io_in_progress = stats.io_in_progress;
+
+ return res;
+}
+
HeapTuple
heap_getnext(TableScanDesc sscan, ScanDirection direction)
{
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 253a735b6c1..51ac2164afd 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2650,6 +2650,7 @@ static const TableAmRoutine heapam_methods = {
.scan_set_tidrange = heap_set_tidrange,
.scan_getnextslot_tidrange = heap_getnextslot_tidrange,
+ .scan_stats = heap_scan_stats,
.parallelscan_estimate = table_block_parallelscan_estimate,
.parallelscan_initialize = table_block_parallelscan_initialize,
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 296ea8a1ed2..ad86fb752ca 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,8 @@
*/
#include "postgres.h"
+#include "access/genam.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +141,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_info(ScanState *planstate,
+ ExplainState *es);
+static void show_worker_io_info(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2016,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_info((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2035,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_info((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2297,8 +2306,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->walusage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
@@ -2315,6 +2324,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+
+ /* show prefetch info for the given worker */
+ show_worker_io_info(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3985,6 +3998,255 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+/*
+ * show_scan_io_info
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_info(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ TableScanStats leader_stats;
+ TableScanStatsData stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started, no prefetch stats */
+ if (!(planstate && planstate->ss_currentScanDesc))
+ return;
+
+ /* collect prefetch statistics from the read stream */
+ leader_stats = table_scan_stats(planstate->ss_currentScanDesc);
+
+ if (leader_stats)
+ {
+ memcpy(&stats, leader_stats, sizeof(TableScanStatsData));
+ }
+ else
+ {
+ memset(&stats, 0, sizeof(TableScanStatsData));
+ }
+
+ /* Initialize counters with stats from the local process first */
+ switch (nodeTag(plan))
+ {
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ stats.prefetch_count += winstrument->stream.prefetch_count;
+ stats.distance_sum += winstrument->stream.distance_sum;
+ if (winstrument->stream.distance_max > stats.distance_max)
+ stats.distance_max = winstrument->stream.distance_max;
+ if (winstrument->stream.distance_capacity > stats.distance_capacity)
+ stats.distance_capacity = winstrument->stream.distance_capacity;
+ stats.stall_count += winstrument->stream.stall_count;
+ stats.io_count += winstrument->stream.io_count;
+ stats.io_nblocks += winstrument->stream.io_nblocks;
+ stats.io_in_progress += winstrument->stream.io_in_progress;
+ }
+ }
+
+ break;
+ }
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ stats.prefetch_count += winstrument->stream.prefetch_count;
+ stats.distance_sum += winstrument->stream.distance_sum;
+ if (winstrument->stream.distance_max > stats.distance_max)
+ stats.distance_max = winstrument->stream.distance_max;
+ if (winstrument->stream.distance_capacity > stats.distance_capacity)
+ stats.distance_capacity = winstrument->stream.distance_capacity;
+ stats.stall_count += winstrument->stream.stall_count;
+ stats.io_count += winstrument->stream.io_count;
+ stats.io_nblocks += winstrument->stream.io_nblocks;
+ stats.io_in_progress += winstrument->stream.io_in_progress;
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ /* don't print anything without prefetching */
+ if (stats.prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats.distance_sum * 1.0 / stats.prefetch_count),
+ stats.distance_max,
+ stats.distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats.stall_count > 0 || stats.io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats.stall_count);
+
+ if (stats.io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats.io_nblocks * 1.0 / stats.io_count),
+ (stats.io_in_progress * 1.0 / stats.io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats.distance_sum * 1.0 / stats.prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats.distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats.distance_capacity, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats.stall_count, es);
+
+ if (stats.io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats.io_nblocks * 1.0 / stats.io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats.io_in_progress * 1.0 / stats.io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+ }
+}
+
+/*
+ * show_io_worker_info
+ * show info about prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ ReadStreamInstrumentation *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stream;
+
+ break;
+ }
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stream;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->stall_count > 0 || stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats->stall_count);
+
+ if (stats->io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats->distance_capacity, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+
+ if (stats->io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+ }
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..f5cbdd21aaa 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -159,6 +159,10 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -185,6 +189,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ac84af294c9..ffc708ed6be 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1119,6 +1119,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7cf8d23c742..bf9af7596ce 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -315,6 +315,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
if (node->sinstrument != NULL && IsParallelWorker())
{
BitmapHeapScanInstrumentation *si;
+ TableScanStats stats;
Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
@@ -328,6 +329,21 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
+ {
+ si->stream.prefetch_count += stats->prefetch_count;
+ si->stream.distance_sum += stats->distance_sum;
+ if (stats->distance_max > si->stream.distance_max)
+ si->stream.distance_max = stats->distance_max;
+ if (stats->distance_capacity > si->stream.distance_capacity)
+ si->stream.distance_capacity = stats->distance_capacity;
+ si->stream.stall_count += stats->stall_count;
+ si->stream.io_count += stats->io_count;
+ si->stream.io_nblocks += stats->io_nblocks;
+ si->stream.io_in_progress += stats->io_in_progress;
+ }
}
/*
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..7992946a5be 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -295,6 +295,43 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+ TableScanStats stats;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
+ {
+ si->stream.prefetch_count += stats->prefetch_count;
+ si->stream.distance_sum += stats->distance_sum;
+ if (stats->distance_max > si->stream.distance_max)
+ si->stream.distance_max = stats->distance_max;
+ if (stats->distance_capacity > si->stream.distance_capacity)
+ si->stream.distance_capacity = stats->distance_capacity;
+ si->stream.stall_count += stats->stall_count;
+ si->stream.io_count += stats->io_count;
+ si->stream.io_nblocks += stats->io_nblocks;
+ si->stream.io_in_progress += stats->io_in_progress;
+ }
+ }
+
/*
* get information from node
*/
@@ -349,10 +386,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -368,14 +418,42 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -404,9 +482,48 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /*
+ * Workers don't get the pscan_len value in scan descriptor, so use the
+ * TAM callback again. The result has to match the earlier result in
+ * ExecSeqScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74ac..f560047afd6 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -107,6 +108,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* stats counters */
+ ReadStreamInstrumentation stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -172,6 +176,38 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * read_stream_update_stats_prefetch
+ * update read_stream stats with current pinned buffer depth
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_update_stats_prefetch(ReadStream *stream)
+{
+ stream->stats.prefetch_count++;
+ stream->stats.distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stream->stats.distance_max)
+ stream->stats.distance_max = stream->pinned_buffers;
+}
+
+/*
+ * read_stream_update_stats_io
+ * update read_stream stats about size of I/O requests
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ stream->stats.io_count++;
+ stream->stats.io_nblocks += nblocks;
+ stream->stats.io_in_progress += in_progress;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +416,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -703,6 +742,10 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
+ /* zero the stats, then set capacity */
+ memset(&stream->stats, 0, sizeof(ReadStreamInstrumentation));
+ stream->stats.distance_capacity = max_pinned_buffers;
+
/*
* Skip the initial ramp-up phase if the caller says we're going to be
* reading the whole relation. This way we start out assuming we'll be
@@ -851,6 +894,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
@@ -860,6 +904,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
}
else
{
@@ -871,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
stream->fast_path = false;
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
#endif
@@ -916,12 +964,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we need to wait for IO */
+ if (needed_wait)
+ stream->stats.stall_count += 1;
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1034,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_update_stats_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
@@ -1118,3 +1173,10 @@ read_stream_end(ReadStream *stream)
read_stream_reset(stream);
pfree(stream);
}
+
+/* return the prefetch stats for the read_stream */
+ReadStreamInstrumentation
+read_stream_prefetch_stats(ReadStream *stream)
+{
+ return stream->stats;
+}
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2fdc50b865b..81dc0b7521d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -359,6 +359,7 @@ extern void heap_prepare_pagescan(TableScanDesc sscan);
extern void heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
bool allow_strat, bool allow_sync, bool allow_pagemode);
extern void heap_endscan(TableScanDesc sscan);
+extern TableScanStats heap_scan_stats(TableScanDesc sscan);
extern HeapTuple heap_getnext(TableScanDesc sscan, ScanDirection direction);
extern bool heap_getnextslot(TableScanDesc sscan,
ScanDirection direction, TupleTableSlot *slot);
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index ce340c076f8..cada9f28bd8 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -114,6 +114,33 @@ typedef struct ParallelBlockTableScanWorkerData
} ParallelBlockTableScanWorkerData;
typedef struct ParallelBlockTableScanWorkerData *ParallelBlockTableScanWorker;
+/*
+ * Generic prefetch stats for table scans.
+ */
+typedef struct TableScanStatsData
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} TableScanStatsData;
+typedef struct TableScanStatsData *TableScanStats;
+
/*
* Base class for fetches from a table via an index. This is the base-class
* for such scans, which needs to be embedded in the respective struct for
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..ab84b76443c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -380,6 +380,11 @@ typedef struct TableAmRoutine
ScanDirection direction,
TupleTableSlot *slot);
+ /*
+ * Collect statistics about table scan.
+ */
+ TableScanStats (*scan_stats) (TableScanDesc scan);
+
/* ------------------------------------------------------------------------
* Parallel table scan related functions.
* ------------------------------------------------------------------------
@@ -1006,6 +1011,18 @@ table_endscan(TableScanDesc scan)
scan->rs_rd->rd_tableam->scan_end(scan);
}
+/*
+ * Fetch statistics about table scan.
+ */
+static inline TableScanStats
+table_scan_stats(TableScanDesc scan)
+{
+ if (scan->rs_rd->rd_tableam->scan_stats)
+ return scan->rs_rd->rd_tableam->scan_stats(scan);
+
+ return NULL;
+}
+
/*
* Restart a relation scan.
*/
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 8847d7f94fa..d94dbbc917d 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -41,9 +41,51 @@ typedef struct SharedAggInfo
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information about read streams
* ---------------------
*/
+typedef struct ReadStreamInstrumentation
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} ReadStreamInstrumentation;
+
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ ReadStreamInstrumentation stream;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +113,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ ReadStreamInstrumentation stream;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 0716c5a9aed..96a8a75031f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1644,6 +1644,8 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SeqScanInstrumentation stats;
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..ec396db5369 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct ReadStreamInstrumentation ReadStreamInstrumentation;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -104,4 +105,6 @@ extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern ReadStreamInstrumentation read_stream_prefetch_stats(ReadStream *stream);
+
#endif /* READ_STREAM_H */
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-17 18:40 ` Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-03-17 18:40 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-03-17 18:51:15 +0100, Tomas Vondra wrote:
> Subject: [PATCH v2 2/2] explain: show prefetch stats in EXPLAIN (ANALYZE,
> VERBOSE)
>
> This adds details about AIO / prefetch for a number of executor nodes
> using the ReadStream, notably:
>
> - SeqScan
> - BitmapHeapScan
>
> The statistics is tracked by the ReadStream, and then propagated up
> through the table AM interface.
>
> The ReadStream tracks the statistics unconditionally, i.e. even outside
> EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
> integer counters), it's not worth gating this by a flag.
>
> The TAM gets one new callback "scan_stats", to collect stats from the
> scan (which fetch tuples from the TAM). There is also a new struct
> TableScanStatsData/TableScanStats to separate the statistics from the
> actual TAM implementation.
It's not really clear to me that this need to go through the tableam
interface. All the accesses already happen within AM code, it seems we could
just populate fields in TableScanDesc TableScanDesc->rs_flags indicates that
that is desired.
That seems like it might be nicer, because
a) there could be multiple read stream instances within a scan
b) we will need one such callback for each different type of scan - so we
could just as well do that within the scan datastructure
Showing any such detail in explain will always be a bit of a piercing of
abstraction layers, I'm not particularly concerned with having optionally
fillable information in *ScanDesc that some AM might not want in that
shape. It seems unlikely that a lot of other architectural decisions would be
based on that, so we can just evolve this in the future.
> + case T_SeqScan:
> + {
> + SharedSeqScanInstrumentation *sinstrument
> + = ((SeqScanState *) planstate)->sinstrument;
> +
> + /* get the sum of the counters set within each and every process */
> + if (sinstrument)
> + {
> + for (int i = 0; i < sinstrument->num_workers; ++i)
> + {
> + SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
> +
> + stats.prefetch_count += winstrument->stream.prefetch_count;
> + stats.distance_sum += winstrument->stream.distance_sum;
> + if (winstrument->stream.distance_max > stats.distance_max)
> + stats.distance_max = winstrument->stream.distance_max;
> + if (winstrument->stream.distance_capacity > stats.distance_capacity)
> + stats.distance_capacity = winstrument->stream.distance_capacity;
> + stats.stall_count += winstrument->stream.stall_count;
> + stats.io_count += winstrument->stream.io_count;
> + stats.io_nblocks += winstrument->stream.io_nblocks;
> + stats.io_in_progress += winstrument->stream.io_in_progress;
> + }
> + }
> +
> + break;
> + }
> + case T_BitmapHeapScan:
> + {
> + SharedBitmapHeapInstrumentation *sinstrument
> + = ((BitmapHeapScanState *) planstate)->sinstrument;
> +
> + /* get the sum of the counters set within each and every process */
> + if (sinstrument)
> + {
> + for (int i = 0; i < sinstrument->num_workers; ++i)
> + {
> + BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
> +
> + stats.prefetch_count += winstrument->stream.prefetch_count;
> + stats.distance_sum += winstrument->stream.distance_sum;
> + if (winstrument->stream.distance_max > stats.distance_max)
> + stats.distance_max = winstrument->stream.distance_max;
> + if (winstrument->stream.distance_capacity > stats.distance_capacity)
> + stats.distance_capacity = winstrument->stream.distance_capacity;
> + stats.stall_count += winstrument->stream.stall_count;
> + stats.io_count += winstrument->stream.io_count;
> + stats.io_nblocks += winstrument->stream.io_nblocks;
> + stats.io_in_progress += winstrument->stream.io_in_progress;
It doesn't seem good to duplicate these fairly large blocks. Can't that be in
a helper?
> + if (es->format == EXPLAIN_FORMAT_TEXT)
> + {
> + /* prefetch distance info */
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
> + (stats.distance_sum * 1.0 / stats.prefetch_count),
> + stats.distance_max,
> + stats.distance_capacity);
> + appendStringInfoChar(es->str, '\n');
> +
> + /* prefetch I/O info (only if there were actual I/Os) */
> + if (stats.stall_count > 0 || stats.io_count > 0)
> + {
Cann there be stalls without IO? Why check both?
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
> + stats.stall_count);
> +
> + if (stats.io_count > 0)
> + {
> + appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
> + (stats.io_nblocks * 1.0 / stats.io_count),
> + (stats.io_in_progress * 1.0 / stats.io_count));
> + }
> +
> + appendStringInfoChar(es->str, '\n');
> + }
> + }
> + else
> + {
> + ExplainOpenGroup("Prefetch", "I/O", true, es);
> +
> + ExplainPropertyFloat("Average Distance", NULL,
> + (stats.distance_sum * 1.0 / stats.prefetch_count), 3, es);
> + ExplainPropertyInteger("Max Distance", NULL,
> + stats.distance_max, es);
> + ExplainPropertyInteger("Capacity", NULL,
> + stats.distance_capacity, es);
> + ExplainPropertyUInteger("Stalls", NULL,
> + stats.stall_count, es);
> +
> + if (stats.io_count > 0)
> + {
> + ExplainPropertyFloat("Average IO Size", NULL,
> + (stats.io_nblocks * 1.0 / stats.io_count), 3, es);
> + ExplainPropertyFloat("Average IOs In Progress", NULL,
> + (stats.io_in_progress * 1.0 / stats.io_count), 3, es);
> + }
> +
> + ExplainCloseGroup("Prefetch", "I/O", true, es);
> + }
> + }
> +}
> +
> +/*
> + * show_io_worker_info
> + * show info about prefetching for a single worker
> + *
> + * Shows prefetching stats for a parallel scan worker.
> + */
> +static void
> +show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
> +{
> + Plan *plan = planstate->plan;
> + ReadStreamInstrumentation *stats = NULL;
> +
> + if (!es->io)
> + return;
> +
> + /* get instrumentation for the given worker */
> + switch (nodeTag(plan))
> + {
> + case T_BitmapHeapScan:
> + {
> + BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
> + SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
> + BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
> +
> + stats = &instrument->stream;
> +
> + break;
> + }
> + case T_SeqScan:
> + {
> + SeqScanState *state = ((SeqScanState *) planstate);
> + SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
> + SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
> +
> + stats = &instrument->stream;
> +
> + break;
> + }
> + default:
> + /* ignore other plans */
> + return;
> + }
> +
> + /* don't print stats if there's nothing to report */
> + if (stats->prefetch_count > 0)
> + {
> + if (es->format == EXPLAIN_FORMAT_TEXT)
> + {
> + /* prefetch distance info */
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
> + (stats->distance_sum * 1.0 / stats->prefetch_count),
> + stats->distance_max,
> + stats->distance_capacity);
> + appendStringInfoChar(es->str, '\n');
> +
> + /* prefetch I/O info (only if there were actual I/Os) */
> + if (stats->stall_count > 0 || stats->io_count > 0)
> + {
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
> + stats->stall_count);
> +
> + if (stats->io_count > 0)
> + {
> + appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
> + (stats->io_nblocks * 1.0 / stats->io_count),
> + (stats->io_in_progress * 1.0 / stats->io_count));
> + }
> +
> + appendStringInfoChar(es->str, '\n');
> + }
> + }
> + else
> + {
> + ExplainOpenGroup("Prefetch", "I/O", true, es);
> +
> + ExplainPropertyFloat("Average Distance", NULL,
> + (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
> + ExplainPropertyInteger("Max Distance", NULL,
> + stats->distance_max, es);
> + ExplainPropertyInteger("Capacity", NULL,
> + stats->distance_capacity, es);
> + ExplainPropertyUInteger("Stalls", NULL,
> + stats->stall_count, es);
> +
> + if (stats->io_count > 0)
> + {
> + ExplainPropertyFloat("Average IO Size", NULL,
> + (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
> + ExplainPropertyFloat("Average IOs In Progress", NULL,
> + (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
> + }
> +
> + ExplainCloseGroup("Prefetch", "I/O", true, es);
> + }
> + }
> +}
Lots of duplication between show_scan_io_info() and show_worker_io_info(). I
assume that's just due to prototype state?
> +/*
> + * read_stream_update_stats_prefetch
> + * update read_stream stats with current pinned buffer depth
Other stuff in read stream does not do this absurd thing of restating the
function name in a comment and indenting the "title" of the function. Don't do
it here.
> + *
> + * Called once per buffer returned to the consumer in read_stream_next_buffer().
> + * Records the number of pinned buffers at that moment, so we can compute the
> + * average look-ahead depth.
> + */
> +static inline void
> +read_stream_update_stats_prefetch(ReadStream *stream)
> +{
> + stream->stats.prefetch_count++;
> + stream->stats.distance_sum += stream->pinned_buffers;
> + if (stream->pinned_buffers > stream->stats.distance_max)
> + stream->stats.distance_max = stream->pinned_buffers;
> +}
> +/*
> + * read_stream_update_stats_io
> + * update read_stream stats about size of I/O requests
> + *
> + * We count the number of I/O requests, size of requests (counted in blocks)
> + * and number of in-progress I/Os.
> + */
> +static inline void
> +read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
> +{
> + stream->stats.io_count++;
> + stream->stats.io_nblocks += nblocks;
> + stream->stats.io_in_progress += in_progress;
> +}
What's the point of the caller passing in stream->ios_in_progress? Are you
expecting some callers to pass in different values? If so, why?
Seems a bit confusing to have both stream->stats.io_in_progress and
streams->ios_in_progress. It's also not clear that stats.io_in_progress is a
sum from the name.
> @@ -851,6 +894,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
> flags)))
> {
> /* Fast return. */
> + read_stream_update_stats_prefetch(stream);
> return buffer;
> }
>
> @@ -860,6 +904,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
> stream->ios_in_progress = 1;
> stream->ios[0].buffer_index = oldest_buffer_index;
> stream->seq_blocknum = next_blocknum + 1;
> +
> + /* update I/O stats */
> + read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
> }
> else
> {
> @@ -871,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
> }
>
> stream->fast_path = false;
> + read_stream_update_stats_prefetch(stream);
> return buffer;
> }
> #endif
Should read_stream_update_stats_prefetch() actually be called when we reached
the end of the stream?
> @@ -916,12 +964,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
> {
> int16 io_index = stream->oldest_io_index;
> int32 distance; /* wider temporary value, clamped below */
> + bool needed_wait;
>
> /* Sanity check that we still agree on the buffers. */
> Assert(stream->ios[io_index].op.buffers ==
> &stream->buffers[oldest_buffer_index]);
>
> - WaitReadBuffers(&stream->ios[io_index].op);
> + needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
> +
> + /* Count it as a stall if we need to wait for IO */
> + if (needed_wait)
> + stream->stats.stall_count += 1;
>
> Assert(stream->ios_in_progress > 0);
> stream->ios_in_progress--;
I'd probably put the stalls_count++ in a helper too, that makes it easier to
test compiling out the stats support and similar things.
> @@ -328,6 +329,21 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
> */
> si->exact_pages += node->stats.exact_pages;
> si->lossy_pages += node->stats.lossy_pages;
> +
> + /* collect prefetch info for this process from the read_stream */
> + if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
> + {
> + si->stream.prefetch_count += stats->prefetch_count;
> + si->stream.distance_sum += stats->distance_sum;
> + if (stats->distance_max > si->stream.distance_max)
> + si->stream.distance_max = stats->distance_max;
> + if (stats->distance_capacity > si->stream.distance_capacity)
> + si->stream.distance_capacity = stats->distance_capacity;
> + si->stream.stall_count += stats->stall_count;
> + si->stream.io_count += stats->io_count;
> + si->stream.io_nblocks += stats->io_nblocks;
> + si->stream.io_in_progress += stats->io_in_progress;
> + }
> }
Could this be wrapped in a helper function?
> +/*
> + * Generic prefetch stats for table scans.
> + */
> +typedef struct TableScanStatsData
> +{
> + /* number of buffers returned to consumer (for averaging distance) */
> + uint64 prefetch_count;
> +
> + /* sum of pinned_buffers sampled at each buffer return */
> + uint64 distance_sum;
> +
> + /* maximum actual pinned_buffers observed during the scan */
> + int16 distance_max;
> +
> + /* maximum possible look-ahead distance (max_pinned_buffers) */
> + int16 distance_capacity;
> +
> + /* number of stalled reads (waiting for I/O) */
> + uint64 stall_count;
> +
> + /* I/O stats */
> + uint64 io_count; /* number of I/Os */
> + uint64 io_nblocks; /* sum of blocks for all I/Os */
> + uint64 io_in_progress; /* sum of in-progress I/Os */
> +} TableScanStatsData;
> +typedef struct TableScanStatsData *TableScanStats;
> --- a/src/include/executor/instrument_node.h
> +++ b/src/include/executor/instrument_node.h
> @@ -41,9 +41,51 @@ typedef struct SharedAggInfo
>
>
> /* ---------------------
> - * Instrumentation information for indexscans (amgettuple and amgetbitmap)
> + * Instrumentation information about read streams
> * ---------------------
> */
> +typedef struct ReadStreamInstrumentation
> +{
> + /* number of buffers returned to consumer (for averaging distance) */
> + uint64 prefetch_count;
> +
> + /* sum of pinned_buffers sampled at each buffer return */
> + uint64 distance_sum;
> +
> + /* maximum actual pinned_buffers observed during the scan */
> + int16 distance_max;
> +
> + /* maximum possible look-ahead distance (max_pinned_buffers) */
> + int16 distance_capacity;
> +
> + /* number of stalled reads (waiting for I/O) */
> + uint64 stall_count;
> +
> + /* I/O stats */
> + uint64 io_count; /* number of I/Os */
> + uint64 io_nblocks; /* sum of blocks for all I/Os */
> + uint64 io_in_progress; /* sum of in-progress I/Os */
> +} ReadStreamInstrumentation;
I don't get what we gain by having this twice, once as TableScanStatsData and
once as ReadStreamInstrumentation.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-03-17 19:32 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-17 19:32 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/17/26 19:40, Andres Freund wrote:
> Hi,
>
> On 2026-03-17 18:51:15 +0100, Tomas Vondra wrote:
>> Subject: [PATCH v2 2/2] explain: show prefetch stats in EXPLAIN (ANALYZE,
>> VERBOSE)
>>
>> This adds details about AIO / prefetch for a number of executor nodes
>> using the ReadStream, notably:
>>
>> - SeqScan
>> - BitmapHeapScan
>>
>> The statistics is tracked by the ReadStream, and then propagated up
>> through the table AM interface.
>>
>> The ReadStream tracks the statistics unconditionally, i.e. even outside
>> EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
>> integer counters), it's not worth gating this by a flag.
>>
>> The TAM gets one new callback "scan_stats", to collect stats from the
>> scan (which fetch tuples from the TAM). There is also a new struct
>> TableScanStatsData/TableScanStats to separate the statistics from the
>> actual TAM implementation.
>
> It's not really clear to me that this need to go through the tableam
> interface. All the accesses already happen within AM code, it seems we could
> just populate fields in TableScanDesc TableScanDesc->rs_flags indicates that
> that is desired.
>
Interesting idea to pass this data through the scan descriptor ...
> That seems like it might be nicer, because
>
> a) there could be multiple read stream instances within a scan
> b) we will need one such callback for each different type of scan - so we
> could just as well do that within the scan datastructure
>
AFAICS (a) is very similar to the argument I made regarding the
stack-based instrumentation earlier, as it wasn't clear to me how would
that work for scans with multiple streams. It didn't occur to me it's an
issue for the current solution with TAM callbacks too.
However, does passing the info through the scan descriptor address this?
I don't see how could it, if there's just a single field per scan.
Also, what do you mean by "one callback for each different type of
scan"? Right now we need two callbacks - one for TableScanDesc and one
for IndexScanDesc. Do we expect to get more?
I don't see much difference between an optional TAM callback for each
scan type, and an optional field in each scan descriptor.
We can base this on rs_flags, but wouldn't it be required in all
table_beginscan variants anyway? I have initially planned to pass a flag
indicating we're in EXPLAIN (ANALYZE), but rs_flags is not that. It's
hard-coded in tableam.
>
> Showing any such detail in explain will always be a bit of a piercing of
> abstraction layers, I'm not particularly concerned with having optionally
> fillable information in *ScanDesc that some AM might not want in that
> shape. It seems unlikely that a lot of other architectural decisions would be
> based on that, so we can just evolve this in the future.
>
>
>> + case T_SeqScan:
>> + {
>> + SharedSeqScanInstrumentation *sinstrument
>> + = ((SeqScanState *) planstate)->sinstrument;
>> +
>> + /* get the sum of the counters set within each and every process */
>> + if (sinstrument)
>> + {
>> + for (int i = 0; i < sinstrument->num_workers; ++i)
>> + {
>> + SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
>> +
>> + stats.prefetch_count += winstrument->stream.prefetch_count;
>> + stats.distance_sum += winstrument->stream.distance_sum;
>> + if (winstrument->stream.distance_max > stats.distance_max)
>> + stats.distance_max = winstrument->stream.distance_max;
>> + if (winstrument->stream.distance_capacity > stats.distance_capacity)
>> + stats.distance_capacity = winstrument->stream.distance_capacity;
>> + stats.stall_count += winstrument->stream.stall_count;
>> + stats.io_count += winstrument->stream.io_count;
>> + stats.io_nblocks += winstrument->stream.io_nblocks;
>> + stats.io_in_progress += winstrument->stream.io_in_progress;
>> + }
>> + }
>> +
>> + break;
>> + }
>> + case T_BitmapHeapScan:
>> + {
>> + SharedBitmapHeapInstrumentation *sinstrument
>> + = ((BitmapHeapScanState *) planstate)->sinstrument;
>> +
>> + /* get the sum of the counters set within each and every process */
>> + if (sinstrument)
>> + {
>> + for (int i = 0; i < sinstrument->num_workers; ++i)
>> + {
>> + BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
>> +
>> + stats.prefetch_count += winstrument->stream.prefetch_count;
>> + stats.distance_sum += winstrument->stream.distance_sum;
>> + if (winstrument->stream.distance_max > stats.distance_max)
>> + stats.distance_max = winstrument->stream.distance_max;
>> + if (winstrument->stream.distance_capacity > stats.distance_capacity)
>> + stats.distance_capacity = winstrument->stream.distance_capacity;
>> + stats.stall_count += winstrument->stream.stall_count;
>> + stats.io_count += winstrument->stream.io_count;
>> + stats.io_nblocks += winstrument->stream.io_nblocks;
>> + stats.io_in_progress += winstrument->stream.io_in_progress;
>
> It doesn't seem good to duplicate these fairly large blocks. Can't that be in
> a helper?
>
Yes, good point.
>
>> + if (es->format == EXPLAIN_FORMAT_TEXT)
>> + {
>> + /* prefetch distance info */
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
>> + (stats.distance_sum * 1.0 / stats.prefetch_count),
>> + stats.distance_max,
>> + stats.distance_capacity);
>> + appendStringInfoChar(es->str, '\n');
>> +
>> + /* prefetch I/O info (only if there were actual I/Os) */
>> + if (stats.stall_count > 0 || stats.io_count > 0)
>> + {
>
> Cann there be stalls without IO? Why check both?
>
I don't recall. I think I originally checked only (io_count > 0), but
then at some point added the other condition. I should have documented
the reason better, I'll check.
>
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
>> + stats.stall_count);
>> +
>> + if (stats.io_count > 0)
>> + {
>> + appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
>> + (stats.io_nblocks * 1.0 / stats.io_count),
>> + (stats.io_in_progress * 1.0 / stats.io_count));
>> + }
>> +
>> + appendStringInfoChar(es->str, '\n');
>> + }
>> + }
>> + else
>> + {
>> + ExplainOpenGroup("Prefetch", "I/O", true, es);
>> +
>> + ExplainPropertyFloat("Average Distance", NULL,
>> + (stats.distance_sum * 1.0 / stats.prefetch_count), 3, es);
>> + ExplainPropertyInteger("Max Distance", NULL,
>> + stats.distance_max, es);
>> + ExplainPropertyInteger("Capacity", NULL,
>> + stats.distance_capacity, es);
>> + ExplainPropertyUInteger("Stalls", NULL,
>> + stats.stall_count, es);
>> +
>> + if (stats.io_count > 0)
>> + {
>> + ExplainPropertyFloat("Average IO Size", NULL,
>> + (stats.io_nblocks * 1.0 / stats.io_count), 3, es);
>> + ExplainPropertyFloat("Average IOs In Progress", NULL,
>> + (stats.io_in_progress * 1.0 / stats.io_count), 3, es);
>> + }
>> +
>> + ExplainCloseGroup("Prefetch", "I/O", true, es);
>> + }
>> + }
>> +}
>> +
>> +/*
>> + * show_io_worker_info
>> + * show info about prefetching for a single worker
>> + *
>> + * Shows prefetching stats for a parallel scan worker.
>> + */
>> +static void
>> +show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
>> +{
>> + Plan *plan = planstate->plan;
>> + ReadStreamInstrumentation *stats = NULL;
>> +
>> + if (!es->io)
>> + return;
>> +
>> + /* get instrumentation for the given worker */
>> + switch (nodeTag(plan))
>> + {
>> + case T_BitmapHeapScan:
>> + {
>> + BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
>> + SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
>> + BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
>> +
>> + stats = &instrument->stream;
>> +
>> + break;
>> + }
>> + case T_SeqScan:
>> + {
>> + SeqScanState *state = ((SeqScanState *) planstate);
>> + SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
>> + SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
>> +
>> + stats = &instrument->stream;
>> +
>> + break;
>> + }
>> + default:
>> + /* ignore other plans */
>> + return;
>> + }
>> +
>> + /* don't print stats if there's nothing to report */
>> + if (stats->prefetch_count > 0)
>> + {
>> + if (es->format == EXPLAIN_FORMAT_TEXT)
>> + {
>> + /* prefetch distance info */
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
>> + (stats->distance_sum * 1.0 / stats->prefetch_count),
>> + stats->distance_max,
>> + stats->distance_capacity);
>> + appendStringInfoChar(es->str, '\n');
>> +
>> + /* prefetch I/O info (only if there were actual I/Os) */
>> + if (stats->stall_count > 0 || stats->io_count > 0)
>> + {
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
>> + stats->stall_count);
>> +
>> + if (stats->io_count > 0)
>> + {
>> + appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
>> + (stats->io_nblocks * 1.0 / stats->io_count),
>> + (stats->io_in_progress * 1.0 / stats->io_count));
>> + }
>> +
>> + appendStringInfoChar(es->str, '\n');
>> + }
>> + }
>> + else
>> + {
>> + ExplainOpenGroup("Prefetch", "I/O", true, es);
>> +
>> + ExplainPropertyFloat("Average Distance", NULL,
>> + (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
>> + ExplainPropertyInteger("Max Distance", NULL,
>> + stats->distance_max, es);
>> + ExplainPropertyInteger("Capacity", NULL,
>> + stats->distance_capacity, es);
>> + ExplainPropertyUInteger("Stalls", NULL,
>> + stats->stall_count, es);
>> +
>> + if (stats->io_count > 0)
>> + {
>> + ExplainPropertyFloat("Average IO Size", NULL,
>> + (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
>> + ExplainPropertyFloat("Average IOs In Progress", NULL,
>> + (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
>> + }
>> +
>> + ExplainCloseGroup("Prefetch", "I/O", true, es);
>> + }
>> + }
>> +}
>
> Lots of duplication between show_scan_io_info() and show_worker_io_info(). I
> assume that's just due to prototype state?
>
Yes, I think it needs helpers as mentioned earlier, etc.
>
>> +/*
>> + * read_stream_update_stats_prefetch
>> + * update read_stream stats with current pinned buffer depth
>
> Other stuff in read stream does not do this absurd thing of restating the
> function name in a comment and indenting the "title" of the function. Don't do
> it here.
>
OK
>
>> + *
>> + * Called once per buffer returned to the consumer in read_stream_next_buffer().
>> + * Records the number of pinned buffers at that moment, so we can compute the
>> + * average look-ahead depth.
>> + */
>> +static inline void
>> +read_stream_update_stats_prefetch(ReadStream *stream)
>> +{
>> + stream->stats.prefetch_count++;
>> + stream->stats.distance_sum += stream->pinned_buffers;
>> + if (stream->pinned_buffers > stream->stats.distance_max)
>> + stream->stats.distance_max = stream->pinned_buffers;
>> +}
>
>
>> +/*
>> + * read_stream_update_stats_io
>> + * update read_stream stats about size of I/O requests
>> + *
>> + * We count the number of I/O requests, size of requests (counted in blocks)
>> + * and number of in-progress I/Os.
>> + */
>> +static inline void
>> +read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
>> +{
>> + stream->stats.io_count++;
>> + stream->stats.io_nblocks += nblocks;
>> + stream->stats.io_in_progress += in_progress;
>> +}
>
> What's the point of the caller passing in stream->ios_in_progress? Are you
> expecting some callers to pass in different values? If so, why?
>
Are you suggesting we remove the argument and just use
stream->ios_in_progress directly? Yes, we can do do that. An earlier
version of the patch did pass (stream->ios_in_progress+1) from one
place, but after that was removed we don't need the argument.
> Seems a bit confusing to have both stream->stats.io_in_progress and
> streams->ios_in_progress. It's also not clear that stats.io_in_progress is a
> sum from the name.
>
I don't think it's all that confusing - those fields are in different
structs. But I agree stream->stats.io_in_progress should indicate it's a
sum of values.
>
>
>> @@ -851,6 +894,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
>> flags)))
>> {
>> /* Fast return. */
>> + read_stream_update_stats_prefetch(stream);
>> return buffer;
>> }
>>
>> @@ -860,6 +904,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
>> stream->ios_in_progress = 1;
>> stream->ios[0].buffer_index = oldest_buffer_index;
>> stream->seq_blocknum = next_blocknum + 1;
>> +
>> + /* update I/O stats */
>> + read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
>> }
>> else
>> {
>> @@ -871,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
>> }
>>
>> stream->fast_path = false;
>> + read_stream_update_stats_prefetch(stream);
>> return buffer;
>> }
>> #endif
>
> Should read_stream_update_stats_prefetch() actually be called when we reached
> the end of the stream?
>
Hmmm, maybe we should not update the stats if
(next_blocknum == InvalidBlockNumber)
>
>
>> @@ -916,12 +964,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
>> {
>> int16 io_index = stream->oldest_io_index;
>> int32 distance; /* wider temporary value, clamped below */
>> + bool needed_wait;
>>
>> /* Sanity check that we still agree on the buffers. */
>> Assert(stream->ios[io_index].op.buffers ==
>> &stream->buffers[oldest_buffer_index]);
>>
>> - WaitReadBuffers(&stream->ios[io_index].op);
>> + needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
>> +
>> + /* Count it as a stall if we need to wait for IO */
>> + if (needed_wait)
>> + stream->stats.stall_count += 1;
>>
>> Assert(stream->ios_in_progress > 0);
>> stream->ios_in_progress--;
>
> I'd probably put the stalls_count++ in a helper too, that makes it easier to
> test compiling out the stats support and similar things.
>
OK, makes sense
>
>
>> @@ -328,6 +329,21 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
>> */
>> si->exact_pages += node->stats.exact_pages;
>> si->lossy_pages += node->stats.lossy_pages;
>> +
>> + /* collect prefetch info for this process from the read_stream */
>> + if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
>> + {
>> + si->stream.prefetch_count += stats->prefetch_count;
>> + si->stream.distance_sum += stats->distance_sum;
>> + if (stats->distance_max > si->stream.distance_max)
>> + si->stream.distance_max = stats->distance_max;
>> + if (stats->distance_capacity > si->stream.distance_capacity)
>> + si->stream.distance_capacity = stats->distance_capacity;
>> + si->stream.stall_count += stats->stall_count;
>> + si->stream.io_count += stats->io_count;
>> + si->stream.io_nblocks += stats->io_nblocks;
>> + si->stream.io_in_progress += stats->io_in_progress;
>> + }
>> }
>
> Could this be wrapped in a helper function?
>
Yes, similarly to the earlier comment.
>
>> +/*
>> + * Generic prefetch stats for table scans.
>> + */
>> +typedef struct TableScanStatsData
>> +{
>> + /* number of buffers returned to consumer (for averaging distance) */
>> + uint64 prefetch_count;
>> +
>> + /* sum of pinned_buffers sampled at each buffer return */
>> + uint64 distance_sum;
>> +
>> + /* maximum actual pinned_buffers observed during the scan */
>> + int16 distance_max;
>> +
>> + /* maximum possible look-ahead distance (max_pinned_buffers) */
>> + int16 distance_capacity;
>> +
>> + /* number of stalled reads (waiting for I/O) */
>> + uint64 stall_count;
>> +
>> + /* I/O stats */
>> + uint64 io_count; /* number of I/Os */
>> + uint64 io_nblocks; /* sum of blocks for all I/Os */
>> + uint64 io_in_progress; /* sum of in-progress I/Os */
>> +} TableScanStatsData;
>> +typedef struct TableScanStatsData *TableScanStats;
>
>
>> --- a/src/include/executor/instrument_node.h
>> +++ b/src/include/executor/instrument_node.h
>> @@ -41,9 +41,51 @@ typedef struct SharedAggInfo
>>
>>
>> /* ---------------------
>> - * Instrumentation information for indexscans (amgettuple and amgetbitmap)
>> + * Instrumentation information about read streams
>> * ---------------------
>> */
>> +typedef struct ReadStreamInstrumentation
>> +{
>> + /* number of buffers returned to consumer (for averaging distance) */
>> + uint64 prefetch_count;
>> +
>> + /* sum of pinned_buffers sampled at each buffer return */
>> + uint64 distance_sum;
>> +
>> + /* maximum actual pinned_buffers observed during the scan */
>> + int16 distance_max;
>> +
>> + /* maximum possible look-ahead distance (max_pinned_buffers) */
>> + int16 distance_capacity;
>> +
>> + /* number of stalled reads (waiting for I/O) */
>> + uint64 stall_count;
>> +
>> + /* I/O stats */
>> + uint64 io_count; /* number of I/Os */
>> + uint64 io_nblocks; /* sum of blocks for all I/Os */
>> + uint64 io_in_progress; /* sum of in-progress I/Os */
>> +} ReadStreamInstrumentation;
>
> I don't get what we gain by having this twice, once as TableScanStatsData and
> once as ReadStreamInstrumentation.
>
Nothing, really. I already suggested we should have just one struct
yesterday.
Thanks!
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-17 20:49 ` Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-03-17 20:49 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; Lukas Fittl <lukas@fittl.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-03-17 20:32:44 +0100, Tomas Vondra wrote:
> On 3/17/26 19:40, Andres Freund wrote:
> > On 2026-03-17 18:51:15 +0100, Tomas Vondra wrote:
> >> Subject: [PATCH v2 2/2] explain: show prefetch stats in EXPLAIN (ANALYZE,
> >> VERBOSE)
> >>
> >> This adds details about AIO / prefetch for a number of executor nodes
> >> using the ReadStream, notably:
> >>
> >> - SeqScan
> >> - BitmapHeapScan
> >>
> >> The statistics is tracked by the ReadStream, and then propagated up
> >> through the table AM interface.
> >>
> >> The ReadStream tracks the statistics unconditionally, i.e. even outside
> >> EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
> >> integer counters), it's not worth gating this by a flag.
> >>
> >> The TAM gets one new callback "scan_stats", to collect stats from the
> >> scan (which fetch tuples from the TAM). There is also a new struct
> >> TableScanStatsData/TableScanStats to separate the statistics from the
> >> actual TAM implementation.
> >
> > It's not really clear to me that this need to go through the tableam
> > interface. All the accesses already happen within AM code, it seems we could
> > just populate fields in TableScanDesc TableScanDesc->rs_flags indicates that
> > that is desired.
> >
>
> Interesting idea to pass this data through the scan descriptor ...
>
> > That seems like it might be nicer, because
> >
> > a) there could be multiple read stream instances within a scan
> > b) we will need one such callback for each different type of scan - so we
> > could just as well do that within the scan datastructure
> >
>
> AFAICS (a) is very similar to the argument I made regarding the
> stack-based instrumentation earlier, as it wasn't clear to me how would
> that work for scans with multiple streams. It didn't occur to me it's an
> issue for the current solution with TAM callbacks too.
>
> However, does passing the info through the scan descriptor address this?
> I don't see how could it, if there's just a single field per scan.
I'd expect the AM to merge the stats internally.
> Also, what do you mean by "one callback for each different type of
> scan"? Right now we need two callbacks - one for TableScanDesc and one
> for IndexScanDesc.
Those two were what I was referring to. The only reason we don't need more is
that we already are combining different scan types (seq, bitmap, tid, sample)
via TableScanDesc, so we really already are quite strongly associating the
stats with the *ScanDesc structures.
Regardless of how we structure it, since we already combine stats for streams
in the parallel query stats, perhaps we should add a field to track for how
many streams the stats are?
> Do we expect to get more?
I think we eventually might want to use read streams for things like getting
the source tuple to update/delete in a bulk update and for detoasting during
expression evaluation (which is surprisingly often IO bound). These would be
bound to an AM but currently we don't have a real descriptor (however, the
need for that has come up repeatedly, e.g. to be able to use multi_insert for
INSERT statements).
I also suspect that there are some executor nodes that could benefit from
doing streaming IO internally, e.g. Sort, Materialize, spilling Hash Agg, Hash
joins... But that'd not be abstracted behind an AM or such, so it's not
really orthogonal.
For things like bulk inserts it would be quite nice if we could use AIO for
reading in index pages with fewer synchronous waits by doing a loop over the
to-be-inserted entries and doing an index lookup that stops at the first miss
and start reading that in.
Before thinking about detoasting I had a bit of a hard time seeing a clear
advantage of using the stack based infra - thinking we always should have very
clear association between the executor node and the stream (in contrast to the
buffer stats which are collected without any possibility to make such an
association). However, if we wanted to e.g. track streams that are done as
part of expression evaluation - I am not at all sure whether we do - that'd
dissolve that clear association, and would be much easier to implement with
the stack based approach.
> I don't see much difference between an optional TAM callback for each
> scan type, and an optional field in each scan descriptor.
It's not entirely different, I agree.
The scan descriptor approach seems a tad nicer because it means that a) the
read stream can be terminated once it's not needed before we get to the stats
collection phase and b) it allows to combine the stats for multiple streams,
without the AM having to store the stats in the AM specific part of the
descriptor, in a field that is then just returned by the AM.
> We can base this on rs_flags, but wouldn't it be required in all
> table_beginscan variants anyway? I have initially planned to pass a flag
> indicating we're in EXPLAIN (ANALYZE), but rs_flags is not that. It's
> hard-coded in tableam.
I think Melanie has a patch that changes that :)
Until then you could just add a helper that ORs in the instrumentation flag.
> >> +/*
> >> + * read_stream_update_stats_io
> >> + * update read_stream stats about size of I/O requests
> >> + *
> >> + * We count the number of I/O requests, size of requests (counted in blocks)
> >> + * and number of in-progress I/Os.
> >> + */
> >> +static inline void
> >> +read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
> >> +{
> >> + stream->stats.io_count++;
> >> + stream->stats.io_nblocks += nblocks;
> >> + stream->stats.io_in_progress += in_progress;
> >> +}
> >
> > What's the point of the caller passing in stream->ios_in_progress? Are you
> > expecting some callers to pass in different values? If so, why?
> >
>
> Are you suggesting we remove the argument and just use
> stream->ios_in_progress directly?
Yes, that's what I am suggesting.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-03-17 21:43 ` Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-17 21:43 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/17/26 21:49, Andres Freund wrote:
> Hi,
>
> On 2026-03-17 20:32:44 +0100, Tomas Vondra wrote:
>> On 3/17/26 19:40, Andres Freund wrote:
>>> On 2026-03-17 18:51:15 +0100, Tomas Vondra wrote:
>>>> Subject: [PATCH v2 2/2] explain: show prefetch stats in EXPLAIN (ANALYZE,
>>>> VERBOSE)
>>>>
>>>> This adds details about AIO / prefetch for a number of executor nodes
>>>> using the ReadStream, notably:
>>>>
>>>> - SeqScan
>>>> - BitmapHeapScan
>>>>
>>>> The statistics is tracked by the ReadStream, and then propagated up
>>>> through the table AM interface.
>>>>
>>>> The ReadStream tracks the statistics unconditionally, i.e. even outside
>>>> EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
>>>> integer counters), it's not worth gating this by a flag.
>>>>
>>>> The TAM gets one new callback "scan_stats", to collect stats from the
>>>> scan (which fetch tuples from the TAM). There is also a new struct
>>>> TableScanStatsData/TableScanStats to separate the statistics from the
>>>> actual TAM implementation.
>>>
>>> It's not really clear to me that this need to go through the tableam
>>> interface. All the accesses already happen within AM code, it seems we could
>>> just populate fields in TableScanDesc TableScanDesc->rs_flags indicates that
>>> that is desired.
>>>
>>
>> Interesting idea to pass this data through the scan descriptor ...
>>
>>> That seems like it might be nicer, because
>>>
>>> a) there could be multiple read stream instances within a scan
>>> b) we will need one such callback for each different type of scan - so we
>>> could just as well do that within the scan datastructure
>>>
>>
>> AFAICS (a) is very similar to the argument I made regarding the
>> stack-based instrumentation earlier, as it wasn't clear to me how would
>> that work for scans with multiple streams. It didn't occur to me it's an
>> issue for the current solution with TAM callbacks too.
>>
>> However, does passing the info through the scan descriptor address this?
>> I don't see how could it, if there's just a single field per scan.
>
> I'd expect the AM to merge the stats internally.
>
Hmmm, I'm not sure that'll be good enough. What if the two streams are
fundamentally different? For example, in the future we may end up using
two streams in index scans. One stream for the table (which is what the
prefetching is doing), and one for the index itself. Does it really make
sense to merge stats for the streams?
Furthermore, which place would be responsible for collecting and merging
the stats? AFAICS it needs to happen at the end of the scan, but it
can't happen in endscan, because that frees the descriptor (so we can't
store the result there). Wouldn't we need a new callback anyway?
>
>> Also, what do you mean by "one callback for each different type of
>> scan"? Right now we need two callbacks - one for TableScanDesc and one
>> for IndexScanDesc.
>
> Those two were what I was referring to. The only reason we don't need more is
> that we already are combining different scan types (seq, bitmap, tid, sample)
> via TableScanDesc, so we really already are quite strongly associating the
> stats with the *ScanDesc structures.
>
> Regardless of how we structure it, since we already combine stats for streams
> in the parallel query stats, perhaps we should add a field to track for how
> many streams the stats are?
>
We could. But does it make sense to combine stats from distinct streams?
>
>> Do we expect to get more?
>
> I think we eventually might want to use read streams for things like getting
> the source tuple to update/delete in a bulk update and for detoasting during
> expression evaluation (which is surprisingly often IO bound). These would be
> bound to an AM but currently we don't have a real descriptor (however, the
> need for that has come up repeatedly, e.g. to be able to use multi_insert for
> INSERT statements).
>
True. But then again - does it make sense to "merge" stats form all
these streams used for very different purposes? OTOH I'm not sure it
works with the TAM approach either, we don't want a separate callback
for each individual stream.
> I also suspect that there are some executor nodes that could benefit from
> doing streaming IO internally, e.g. Sort, Materialize, spilling Hash Agg, Hash
> joins... But that'd not be abstracted behind an AM or such, so it's not
> really orthogonal.
>
Agreed.
>
> For things like bulk inserts it would be quite nice if we could use AIO for
> reading in index pages with fewer synchronous waits by doing a loop over the
> to-be-inserted entries and doing an index lookup that stops at the first miss
> and start reading that in.
>
Yes. I had a very ugly PoC last year to prefetch leaf pages for bulk
inserts, and it was a huge benefit.
>
> Before thinking about detoasting I had a bit of a hard time seeing a clear
> advantage of using the stack based infra - thinking we always should have very
> clear association between the executor node and the stream (in contrast to the
> buffer stats which are collected without any possibility to make such an
> association). However, if we wanted to e.g. track streams that are done as
> part of expression evaluation - I am not at all sure whether we do - that'd
> dissolve that clear association, and would be much easier to implement with
> the stack based approach.
>
Right. In some cases the association to streams may not be quite clear.
Would it be possible for scans to "define" a list of streams? For
example, an index scan have one stream for the index, one stream for
table, one stream for TOAST. And we'd have an array of stats, with one
slot for each stream.
With the current approach, we'd collect stats from each of all those
streams into the correct "slot" in the instrumentation.
With the stack-based instrumentation we might "tell" each stream which
slot to use. So ReadStream would get a field "stats_index" and it'd be
passed to INSTR_IOUSAGE_INCR() to update the correct slot.
I haven't tried any of this, and AFAIK we don't even have a scan using
two read streams yet.
>
>> I don't see much difference between an optional TAM callback for each
>> scan type, and an optional field in each scan descriptor.
>
> It's not entirely different, I agree.
>
> The scan descriptor approach seems a tad nicer because it means that a) the
> read stream can be terminated once it's not needed before we get to the stats
> collection phase and b) it allows to combine the stats for multiple streams,
> without the AM having to store the stats in the AM specific part of the
> descriptor, in a field that is then just returned by the AM.
>
>
>> We can base this on rs_flags, but wouldn't it be required in all
>> table_beginscan variants anyway? I have initially planned to pass a flag
>> indicating we're in EXPLAIN (ANALYZE), but rs_flags is not that. It's
>> hard-coded in tableam.
>
> I think Melanie has a patch that changes that :)
>
> Until then you could just add a helper that ORs in the instrumentation flag.
>
OK.
>
>
>>>> +/*
>>>> + * read_stream_update_stats_io
>>>> + * update read_stream stats about size of I/O requests
>>>> + *
>>>> + * We count the number of I/O requests, size of requests (counted in blocks)
>>>> + * and number of in-progress I/Os.
>>>> + */
>>>> +static inline void
>>>> +read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
>>>> +{
>>>> + stream->stats.io_count++;
>>>> + stream->stats.io_nblocks += nblocks;
>>>> + stream->stats.io_in_progress += in_progress;
>>>> +}
>>>
>>> What's the point of the caller passing in stream->ios_in_progress? Are you
>>> expecting some callers to pass in different values? If so, why?
>>>
>>
>> Are you suggesting we remove the argument and just use
>> stream->ios_in_progress directly?
>
> Yes, that's what I am suggesting.
>
OK, will do.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-18 22:41 ` Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-18 22:41 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi!
Here's a v3 of the patch, adopting the idea of passing the data through
a field in the scan descriptor. I've kept it broken into multiple
smaller pieces 0002-0006 to show the incremental changes, but in the end
this should be all merged.
0001 - a prerequisite patch
0002 - v2 of the patch
0003 - basic cleanup / helpers to reduce code duplication, various
smaller fixes and improvements, ...
0004 - changes the design to use a new field in TableScanDesc
0005 - minor cleanup of 0004
0006 - support for TidRangeScan (the last scan using ReadStream),
requires changes similar to SeqScan in v2
Overall, I think it looks reasonable. There are probably ways to
simplify it further (e.g. by introducing a new read_stream_begin_
helper, so that it's not necessary to update every caller of
read_stream_begin_relation by adding a NULL argument).
The 0003 also changes the EXPLAIN to enable IO by default, just like we
do for BUFFERS. It seems like a reasonable precedent to me.
0004 reworks the tracking to use a field in TableScanDesc, instead of
adding a table AM callback. I agree it seems simpler / less disruptive.
0005 is a minor cleanup for 0004
0006 adds EXPLAIN support for TidRangeScan, similarly to how 0002 adds
support for SeqScan (but that still uses the TAM interface)
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v3-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch (2.8K, ../../bf1d4799-e2fd-464c-96ff-dc448f021b1e@vondra.me/2-v3-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch)
download | inline diff:
From ad614a659df1742240b34212da7f9669b586d623 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH v3 1/6] bufmgr: Return whether WaitReadBuffers() needed to
wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 00bc609529a..c8ec7fb4f53 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1738,12 +1738,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1805,6 +1813,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1835,6 +1844,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1851,6 +1866,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4017896f951..a1ef04354dd 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -249,7 +249,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.53.0
[text/x-patch] v3-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE-VE.patch (35.1K, ../../bf1d4799-e2fd-464c-96ff-dc448f021b1e@vondra.me/3-v3-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE-VE.patch)
download | inline diff:
From e4b049b1db70480d86c6dd8eca398ff6211d7a84 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Thu, 12 Mar 2026 02:04:07 +0100
Subject: [PATCH v3 2/6] explain: show prefetch stats in EXPLAIN (ANALYZE,
VERBOSE)
This adds details about AIO / prefetch for a number of executor nodes
using the ReadStream, notably:
- SeqScan
- BitmapHeapScan
The statistics is tracked by the ReadStream, and then propagated up
through the table AM interface.
The ReadStream tracks the statistics unconditionally, i.e. even outside
EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
integer counters), it's not worth gating this by a flag.
The TAM gets one new callback "scan_stats", to collect stats from the
scan (which fetch tuples from the TAM). There is also a new struct
TableScanStatsData/TableScanStats to separate the statistics from the
actual TAM implementation.
This required improving SeqScan to properly setup shared instrumentation
between the leader and workers, collect it at the end of a scan, etc.
---
doc/src/sgml/ref/explain.sgml | 12 +
src/backend/access/heap/heapam.c | 32 +++
src/backend/access/heap/heapam_handler.c | 1 +
src/backend/commands/explain.c | 266 +++++++++++++++++++++-
src/backend/commands/explain_state.c | 10 +
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeBitmapHeapscan.c | 16 ++
src/backend/executor/nodeSeqscan.c | 123 +++++++++-
src/backend/storage/aio/read_stream.c | 64 +++++-
src/include/access/heapam.h | 1 +
src/include/access/relscan.h | 27 +++
src/include/access/tableam.h | 17 ++
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 45 +++-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 2 +
src/include/storage/read_stream.h | 3 +
17 files changed, 617 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 7dee77fd366..9da16c77d73 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -295,6 +296,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>FALSE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e5bd062de77..b13f843cfad 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1416,6 +1416,38 @@ heap_endscan(TableScanDesc sscan)
pfree(scan);
}
+/*
+ * heap_scan_stats
+ * return stats collected by the read stream
+ *
+ * Returns NULL if the scan is not using a read stream.
+ */
+TableScanStats
+heap_scan_stats(TableScanDesc sscan)
+{
+ HeapScanDesc scan = (HeapScanDesc) sscan;
+ ReadStreamInstrumentation stats;
+ TableScanStats res;
+
+ if (!scan->rs_read_stream)
+ return NULL;
+
+ stats = read_stream_prefetch_stats(scan->rs_read_stream);
+
+ res = palloc0(sizeof(TableScanStatsData));
+
+ res->prefetch_count = stats.prefetch_count;
+ res->distance_sum = stats.distance_sum;
+ res->distance_max = stats.distance_max;
+ res->distance_capacity = stats.distance_capacity;
+ res->stall_count = stats.stall_count;
+ res->io_count = stats.io_count;
+ res->io_nblocks = stats.io_nblocks;
+ res->io_in_progress = stats.io_in_progress;
+
+ return res;
+}
+
HeapTuple
heap_getnext(TableScanDesc sscan, ScanDirection direction)
{
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 253a735b6c1..51ac2164afd 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2650,6 +2650,7 @@ static const TableAmRoutine heapam_methods = {
.scan_set_tidrange = heap_set_tidrange,
.scan_getnextslot_tidrange = heap_getnextslot_tidrange,
+ .scan_stats = heap_scan_stats,
.parallelscan_estimate = table_block_parallelscan_estimate,
.parallelscan_initialize = table_block_parallelscan_initialize,
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 296ea8a1ed2..ad86fb752ca 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,8 @@
*/
#include "postgres.h"
+#include "access/genam.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +141,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_info(ScanState *planstate,
+ ExplainState *es);
+static void show_worker_io_info(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2016,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_info((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2035,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_info((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2297,8 +2306,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->walusage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
@@ -2315,6 +2324,10 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+
+ /* show prefetch info for the given worker */
+ show_worker_io_info(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3985,6 +3998,255 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+/*
+ * show_scan_io_info
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_info(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ TableScanStats leader_stats;
+ TableScanStatsData stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started, no prefetch stats */
+ if (!(planstate && planstate->ss_currentScanDesc))
+ return;
+
+ /* collect prefetch statistics from the read stream */
+ leader_stats = table_scan_stats(planstate->ss_currentScanDesc);
+
+ if (leader_stats)
+ {
+ memcpy(&stats, leader_stats, sizeof(TableScanStatsData));
+ }
+ else
+ {
+ memset(&stats, 0, sizeof(TableScanStatsData));
+ }
+
+ /* Initialize counters with stats from the local process first */
+ switch (nodeTag(plan))
+ {
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ stats.prefetch_count += winstrument->stream.prefetch_count;
+ stats.distance_sum += winstrument->stream.distance_sum;
+ if (winstrument->stream.distance_max > stats.distance_max)
+ stats.distance_max = winstrument->stream.distance_max;
+ if (winstrument->stream.distance_capacity > stats.distance_capacity)
+ stats.distance_capacity = winstrument->stream.distance_capacity;
+ stats.stall_count += winstrument->stream.stall_count;
+ stats.io_count += winstrument->stream.io_count;
+ stats.io_nblocks += winstrument->stream.io_nblocks;
+ stats.io_in_progress += winstrument->stream.io_in_progress;
+ }
+ }
+
+ break;
+ }
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ stats.prefetch_count += winstrument->stream.prefetch_count;
+ stats.distance_sum += winstrument->stream.distance_sum;
+ if (winstrument->stream.distance_max > stats.distance_max)
+ stats.distance_max = winstrument->stream.distance_max;
+ if (winstrument->stream.distance_capacity > stats.distance_capacity)
+ stats.distance_capacity = winstrument->stream.distance_capacity;
+ stats.stall_count += winstrument->stream.stall_count;
+ stats.io_count += winstrument->stream.io_count;
+ stats.io_nblocks += winstrument->stream.io_nblocks;
+ stats.io_in_progress += winstrument->stream.io_in_progress;
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ /* don't print anything without prefetching */
+ if (stats.prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats.distance_sum * 1.0 / stats.prefetch_count),
+ stats.distance_max,
+ stats.distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats.stall_count > 0 || stats.io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats.stall_count);
+
+ if (stats.io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats.io_nblocks * 1.0 / stats.io_count),
+ (stats.io_in_progress * 1.0 / stats.io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats.distance_sum * 1.0 / stats.prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats.distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats.distance_capacity, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats.stall_count, es);
+
+ if (stats.io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats.io_nblocks * 1.0 / stats.io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats.io_in_progress * 1.0 / stats.io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+ }
+}
+
+/*
+ * show_io_worker_info
+ * show info about prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ ReadStreamInstrumentation *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stream;
+
+ break;
+ }
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stream;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->stall_count > 0 || stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats->stall_count);
+
+ if (stats->io_count > 0)
+ {
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "I/O", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats->distance_capacity, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+
+ if (stats->io_count > 0)
+ {
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+ }
+
+ ExplainCloseGroup("Prefetch", "I/O", true, es);
+ }
+ }
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..f5cbdd21aaa 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -159,6 +159,10 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -185,6 +189,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ac84af294c9..ffc708ed6be 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1119,6 +1119,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7cf8d23c742..bf9af7596ce 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -315,6 +315,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
if (node->sinstrument != NULL && IsParallelWorker())
{
BitmapHeapScanInstrumentation *si;
+ TableScanStats stats;
Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
@@ -328,6 +329,21 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
+ {
+ si->stream.prefetch_count += stats->prefetch_count;
+ si->stream.distance_sum += stats->distance_sum;
+ if (stats->distance_max > si->stream.distance_max)
+ si->stream.distance_max = stats->distance_max;
+ if (stats->distance_capacity > si->stream.distance_capacity)
+ si->stream.distance_capacity = stats->distance_capacity;
+ si->stream.stall_count += stats->stall_count;
+ si->stream.io_count += stats->io_count;
+ si->stream.io_nblocks += stats->io_nblocks;
+ si->stream.io_in_progress += stats->io_in_progress;
+ }
}
/*
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..7992946a5be 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -295,6 +295,43 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+ TableScanStats stats;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
+ {
+ si->stream.prefetch_count += stats->prefetch_count;
+ si->stream.distance_sum += stats->distance_sum;
+ if (stats->distance_max > si->stream.distance_max)
+ si->stream.distance_max = stats->distance_max;
+ if (stats->distance_capacity > si->stream.distance_capacity)
+ si->stream.distance_capacity = stats->distance_capacity;
+ si->stream.stall_count += stats->stall_count;
+ si->stream.io_count += stats->io_count;
+ si->stream.io_nblocks += stats->io_nblocks;
+ si->stream.io_in_progress += stats->io_in_progress;
+ }
+ }
+
/*
* get information from node
*/
@@ -349,10 +386,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -368,14 +418,42 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -404,9 +482,48 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /*
+ * Workers don't get the pscan_len value in scan descriptor, so use the
+ * TAM callback again. The result has to match the earlier result in
+ * ExecSeqScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74ac..f560047afd6 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -107,6 +108,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* stats counters */
+ ReadStreamInstrumentation stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -172,6 +176,38 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * read_stream_update_stats_prefetch
+ * update read_stream stats with current pinned buffer depth
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_update_stats_prefetch(ReadStream *stream)
+{
+ stream->stats.prefetch_count++;
+ stream->stats.distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stream->stats.distance_max)
+ stream->stats.distance_max = stream->pinned_buffers;
+}
+
+/*
+ * read_stream_update_stats_io
+ * update read_stream stats about size of I/O requests
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ stream->stats.io_count++;
+ stream->stats.io_nblocks += nblocks;
+ stream->stats.io_in_progress += in_progress;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +416,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -703,6 +742,10 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
+ /* zero the stats, then set capacity */
+ memset(&stream->stats, 0, sizeof(ReadStreamInstrumentation));
+ stream->stats.distance_capacity = max_pinned_buffers;
+
/*
* Skip the initial ramp-up phase if the caller says we're going to be
* reading the whole relation. This way we start out assuming we'll be
@@ -851,6 +894,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
@@ -860,6 +904,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
}
else
{
@@ -871,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
stream->fast_path = false;
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
#endif
@@ -916,12 +964,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we need to wait for IO */
+ if (needed_wait)
+ stream->stats.stall_count += 1;
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1034,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_update_stats_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
@@ -1118,3 +1173,10 @@ read_stream_end(ReadStream *stream)
read_stream_reset(stream);
pfree(stream);
}
+
+/* return the prefetch stats for the read_stream */
+ReadStreamInstrumentation
+read_stream_prefetch_stats(ReadStream *stream)
+{
+ return stream->stats;
+}
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2fdc50b865b..81dc0b7521d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -359,6 +359,7 @@ extern void heap_prepare_pagescan(TableScanDesc sscan);
extern void heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
bool allow_strat, bool allow_sync, bool allow_pagemode);
extern void heap_endscan(TableScanDesc sscan);
+extern TableScanStats heap_scan_stats(TableScanDesc sscan);
extern HeapTuple heap_getnext(TableScanDesc sscan, ScanDirection direction);
extern bool heap_getnextslot(TableScanDesc sscan,
ScanDirection direction, TupleTableSlot *slot);
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index ce340c076f8..cada9f28bd8 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -114,6 +114,33 @@ typedef struct ParallelBlockTableScanWorkerData
} ParallelBlockTableScanWorkerData;
typedef struct ParallelBlockTableScanWorkerData *ParallelBlockTableScanWorker;
+/*
+ * Generic prefetch stats for table scans.
+ */
+typedef struct TableScanStatsData
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} TableScanStatsData;
+typedef struct TableScanStatsData *TableScanStats;
+
/*
* Base class for fetches from a table via an index. This is the base-class
* for such scans, which needs to be embedded in the respective struct for
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..ab84b76443c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -380,6 +380,11 @@ typedef struct TableAmRoutine
ScanDirection direction,
TupleTableSlot *slot);
+ /*
+ * Collect statistics about table scan.
+ */
+ TableScanStats (*scan_stats) (TableScanDesc scan);
+
/* ------------------------------------------------------------------------
* Parallel table scan related functions.
* ------------------------------------------------------------------------
@@ -1006,6 +1011,18 @@ table_endscan(TableScanDesc scan)
scan->rs_rd->rd_tableam->scan_end(scan);
}
+/*
+ * Fetch statistics about table scan.
+ */
+static inline TableScanStats
+table_scan_stats(TableScanDesc scan)
+{
+ if (scan->rs_rd->rd_tableam->scan_stats)
+ return scan->rs_rd->rd_tableam->scan_stats(scan);
+
+ return NULL;
+}
+
/*
* Restart a relation scan.
*/
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 8847d7f94fa..d94dbbc917d 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -41,9 +41,51 @@ typedef struct SharedAggInfo
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information about read streams
* ---------------------
*/
+typedef struct ReadStreamInstrumentation
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} ReadStreamInstrumentation;
+
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ ReadStreamInstrumentation stream;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +113,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ ReadStreamInstrumentation stream;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 0716c5a9aed..96a8a75031f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1644,6 +1644,8 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SeqScanInstrumentation stats;
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..ec396db5369 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct ReadStreamInstrumentation ReadStreamInstrumentation;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -104,4 +105,6 @@ extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern ReadStreamInstrumentation read_stream_prefetch_stats(ReadStream *stream);
+
#endif /* READ_STREAM_H */
--
2.53.0
[text/x-patch] v3-0003-cleanup-and-simplification.patch (62.9K, ../../bf1d4799-e2fd-464c-96ff-dc448f021b1e@vondra.me/4-v3-0003-cleanup-and-simplification.patch)
download | inline diff:
From f278bd14ca6fcd27b02ebb14feed797d0da721ad Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 17 Mar 2026 22:06:46 +0100
Subject: [PATCH v3 3/6] cleanup and simplification
- Reduce code duplication in explain.c by introducing a couple helper
methods for repeated blocks - ACCUMULATE_TABLE_STATS() and
show_io_info().
- Replace ReadStreamInstrumentation with generic IOStats.
- new read_stream_update_stats_stall() wrapper
- enable EXPLAIN (IO) by default, just like BUFFERS
- fix the explain group to have matching objtype/label
- don't update prefetch stats after stream ended
fixup: track prefetch distance
---
doc/src/sgml/ref/explain.sgml | 2 +-
src/backend/access/heap/heapam.c | 2 +-
src/backend/commands/explain.c | 242 +++++++-----------
src/backend/commands/explain_state.c | 5 +
src/backend/executor/nodeBitmapHeapscan.c | 11 +-
src/backend/executor/nodeSeqscan.c | 11 +-
src/backend/storage/aio/read_stream.c | 26 +-
src/include/access/relscan.h | 14 +
src/include/executor/instrument_node.h | 8 +-
src/include/storage/read_stream.h | 4 +-
src/test/regress/expected/explain.out | 39 ++-
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 75 ++++--
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 24 +-
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
24 files changed, 252 insertions(+), 245 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 9da16c77d73..83f27badfc8 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -302,7 +302,7 @@ ROLLBACK;
<para>
Include information on I/O performed by each node.
This parameter may only be used when <literal>ANALYZE</literal> is also
- enabled. It defaults to <literal>FALSE</literal>.
+ enabled. It defaults to <literal>TRUE</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b13f843cfad..323badd27f1 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1426,7 +1426,7 @@ TableScanStats
heap_scan_stats(TableScanDesc sscan)
{
HeapScanDesc scan = (HeapScanDesc) sscan;
- ReadStreamInstrumentation stats;
+ IOStats stats;
TableScanStats res;
if (!scan->rs_read_stream)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ad86fb752ca..7a4009ae74f 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -141,11 +141,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
-static void show_scan_io_info(ScanState *planstate,
- ExplainState *es);
-static void show_worker_io_info(PlanState *planstate,
- ExplainState *es,
- int worker);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
+static void show_io_usage(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2016,7 +2016,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
- show_scan_io_info((ScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2035,7 +2035,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
- show_scan_io_info((ScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2324,9 +2324,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
-
- /* show prefetch info for the given worker */
- show_worker_io_info(planstate, es, n);
+ if (es->io)
+ show_io_usage(planstate, es, n);
ExplainCloseWorker(n, es);
}
@@ -3998,14 +3997,74 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, TableScanStats stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats->stall_count);
+
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "Prefetch", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainCloseGroup("Prefetch", "Prefetch", true, es);
+
+ if (stats->io_count > 0)
+ {
+ ExplainOpenGroup("I/O", "I/O", true, es);
+
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+
+ ExplainCloseGroup("I/O", "I/O", true, es);
+ }
+ }
+ }
+}
+
/*
- * show_scan_io_info
+ * show_scan_io_usage
* show info about prefetching for a seq/bitmap scan
*
* Shows summary of stats for leader and workers (if any).
*/
static void
-show_scan_io_info(ScanState *planstate, ExplainState *es)
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
{
Plan *plan = planstate->ps.plan;
TableScanStats leader_stats;
@@ -4044,17 +4103,7 @@ show_scan_io_info(ScanState *planstate, ExplainState *es)
for (int i = 0; i < sinstrument->num_workers; ++i)
{
SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
-
- stats.prefetch_count += winstrument->stream.prefetch_count;
- stats.distance_sum += winstrument->stream.distance_sum;
- if (winstrument->stream.distance_max > stats.distance_max)
- stats.distance_max = winstrument->stream.distance_max;
- if (winstrument->stream.distance_capacity > stats.distance_capacity)
- stats.distance_capacity = winstrument->stream.distance_capacity;
- stats.stall_count += winstrument->stream.stall_count;
- stats.io_count += winstrument->stream.io_count;
- stats.io_nblocks += winstrument->stream.io_nblocks;
- stats.io_in_progress += winstrument->stream.io_in_progress;
+ ACCUMULATE_TABLE_STATS(&stats, &winstrument->io);
}
}
@@ -4071,17 +4120,7 @@ show_scan_io_info(ScanState *planstate, ExplainState *es)
for (int i = 0; i < sinstrument->num_workers; ++i)
{
BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
-
- stats.prefetch_count += winstrument->stream.prefetch_count;
- stats.distance_sum += winstrument->stream.distance_sum;
- if (winstrument->stream.distance_max > stats.distance_max)
- stats.distance_max = winstrument->stream.distance_max;
- if (winstrument->stream.distance_capacity > stats.distance_capacity)
- stats.distance_capacity = winstrument->stream.distance_capacity;
- stats.stall_count += winstrument->stream.stall_count;
- stats.io_count += winstrument->stream.io_count;
- stats.io_nblocks += winstrument->stream.io_nblocks;
- stats.io_in_progress += winstrument->stream.io_in_progress;
+ ACCUMULATE_TABLE_STATS(&stats, &winstrument->io);
}
}
@@ -4092,73 +4131,23 @@ show_scan_io_info(ScanState *planstate, ExplainState *es)
return;
}
- /* don't print anything without prefetching */
- if (stats.prefetch_count > 0)
- {
- if (es->format == EXPLAIN_FORMAT_TEXT)
- {
- /* prefetch distance info */
- ExplainIndentText(es);
- appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
- (stats.distance_sum * 1.0 / stats.prefetch_count),
- stats.distance_max,
- stats.distance_capacity);
- appendStringInfoChar(es->str, '\n');
-
- /* prefetch I/O info (only if there were actual I/Os) */
- if (stats.stall_count > 0 || stats.io_count > 0)
- {
- ExplainIndentText(es);
- appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
- stats.stall_count);
-
- if (stats.io_count > 0)
- {
- appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
- (stats.io_nblocks * 1.0 / stats.io_count),
- (stats.io_in_progress * 1.0 / stats.io_count));
- }
-
- appendStringInfoChar(es->str, '\n');
- }
- }
- else
- {
- ExplainOpenGroup("Prefetch", "I/O", true, es);
-
- ExplainPropertyFloat("Average Distance", NULL,
- (stats.distance_sum * 1.0 / stats.prefetch_count), 3, es);
- ExplainPropertyInteger("Max Distance", NULL,
- stats.distance_max, es);
- ExplainPropertyInteger("Capacity", NULL,
- stats.distance_capacity, es);
- ExplainPropertyUInteger("Stalls", NULL,
- stats.stall_count, es);
-
- if (stats.io_count > 0)
- {
- ExplainPropertyFloat("Average IO Size", NULL,
- (stats.io_nblocks * 1.0 / stats.io_count), 3, es);
- ExplainPropertyFloat("Average IOs In Progress", NULL,
- (stats.io_in_progress * 1.0 / stats.io_count), 3, es);
- }
-
- ExplainCloseGroup("Prefetch", "I/O", true, es);
- }
- }
+ print_io_usage(es, &stats);
}
/*
- * show_io_worker_info
- * show info about prefetching for a single worker
+ * show_io_usage
+ * show info about I/O prefetching for a single worker
*
* Shows prefetching stats for a parallel scan worker.
*/
static void
-show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
+show_io_usage(PlanState *planstate, ExplainState *es, int worker)
{
Plan *plan = planstate->plan;
- ReadStreamInstrumentation *stats = NULL;
+ IOStats *stats = NULL;
+
+ // XXX
+ TableScanStatsData tmp;
if (!es->io)
return;
@@ -4172,7 +4161,7 @@ show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
- stats = &instrument->stream;
+ stats = &instrument->io;
break;
}
@@ -4182,7 +4171,7 @@ show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
- stats = &instrument->stream;
+ stats = &instrument->io;
break;
}
@@ -4191,60 +4180,17 @@ show_worker_io_info(PlanState *planstate, ExplainState *es, int worker)
return;
}
- /* don't print stats if there's nothing to report */
- if (stats->prefetch_count > 0)
- {
- if (es->format == EXPLAIN_FORMAT_TEXT)
- {
- /* prefetch distance info */
- ExplainIndentText(es);
- appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
- (stats->distance_sum * 1.0 / stats->prefetch_count),
- stats->distance_max,
- stats->distance_capacity);
- appendStringInfoChar(es->str, '\n');
-
- /* prefetch I/O info (only if there were actual I/Os) */
- if (stats->stall_count > 0 || stats->io_count > 0)
- {
- ExplainIndentText(es);
- appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
- stats->stall_count);
-
- if (stats->io_count > 0)
- {
- appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
- (stats->io_nblocks * 1.0 / stats->io_count),
- (stats->io_in_progress * 1.0 / stats->io_count));
- }
-
- appendStringInfoChar(es->str, '\n');
- }
- }
- else
- {
- ExplainOpenGroup("Prefetch", "I/O", true, es);
-
- ExplainPropertyFloat("Average Distance", NULL,
- (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
- ExplainPropertyInteger("Max Distance", NULL,
- stats->distance_max, es);
- ExplainPropertyInteger("Capacity", NULL,
- stats->distance_capacity, es);
- ExplainPropertyUInteger("Stalls", NULL,
- stats->stall_count, es);
-
- if (stats->io_count > 0)
- {
- ExplainPropertyFloat("Average IO Size", NULL,
- (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
- ExplainPropertyFloat("Average IOs In Progress", NULL,
- (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
- }
-
- ExplainCloseGroup("Prefetch", "I/O", true, es);
- }
- }
+ /* XXX convert to TableScanStats */
+ tmp.distance_capacity = stats->distance_capacity;
+ tmp.distance_max = stats->distance_max;
+ tmp.distance_sum = stats->distance_sum;
+ tmp.io_count = stats->io_count;
+ tmp.io_in_progress = stats->io_in_progress;
+ tmp.io_nblocks = stats->io_nblocks;
+ tmp.prefetch_count = stats->prefetch_count;
+ tmp.stall_count = stats->stall_count;
+
+ print_io_usage(es, &tmp);
}
/*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index f5cbdd21aaa..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -161,6 +162,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
}
else if (strcmp(opt->defname, "io") == 0)
{
+ io_set = true;
es->io = defGetBoolean(opt);
}
else if (!ApplyExtensionExplainOption(es, opt, pstate))
@@ -183,6 +185,9 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index bf9af7596ce..0f1459f11a4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -333,16 +333,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/* collect prefetch info for this process from the read_stream */
if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
{
- si->stream.prefetch_count += stats->prefetch_count;
- si->stream.distance_sum += stats->distance_sum;
- if (stats->distance_max > si->stream.distance_max)
- si->stream.distance_max = stats->distance_max;
- if (stats->distance_capacity > si->stream.distance_capacity)
- si->stream.distance_capacity = stats->distance_capacity;
- si->stream.stall_count += stats->stall_count;
- si->stream.io_count += stats->io_count;
- si->stream.io_nblocks += stats->io_nblocks;
- si->stream.io_in_progress += stats->io_in_progress;
+ ACCUMULATE_TABLE_STATS(stats, &si->io);
}
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 7992946a5be..4fa85a1b6bc 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -319,16 +319,7 @@ ExecEndSeqScan(SeqScanState *node)
/* collect prefetch info for this process from the read_stream */
if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
{
- si->stream.prefetch_count += stats->prefetch_count;
- si->stream.distance_sum += stats->distance_sum;
- if (stats->distance_max > si->stream.distance_max)
- si->stream.distance_max = stats->distance_max;
- if (stats->distance_capacity > si->stream.distance_capacity)
- si->stream.distance_capacity = stats->distance_capacity;
- si->stream.stall_count += stats->stall_count;
- si->stream.io_count += stats->io_count;
- si->stream.io_nblocks += stats->io_nblocks;
- si->stream.io_in_progress += stats->io_in_progress;
+ ACCUMULATE_TABLE_STATS(stats, &si->io);
}
}
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index f560047afd6..faa52ca4b2c 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -108,8 +108,8 @@ struct ReadStream
bool advice_enabled;
bool temporary;
- /* stats counters */
- ReadStreamInstrumentation stats;
+ /* scan stats counters */
+ IOStats stats;
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
@@ -177,8 +177,7 @@ block_range_read_stream_cb(ReadStream *stream,
}
/*
- * read_stream_update_stats_prefetch
- * update read_stream stats with current pinned buffer depth
+ * Update stream stats with current pinned buffer depth.
*
* Called once per buffer returned to the consumer in read_stream_next_buffer().
* Records the number of pinned buffers at that moment, so we can compute the
@@ -194,8 +193,7 @@ read_stream_update_stats_prefetch(ReadStream *stream)
}
/*
- * read_stream_update_stats_io
- * update read_stream stats about size of I/O requests
+ * Update stream stats about size of I/O requests.
*
* We count the number of I/O requests, size of requests (counted in blocks)
* and number of in-progress I/Os.
@@ -208,6 +206,12 @@ read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
stream->stats.io_in_progress += in_progress;
}
+static inline void
+read_stream_update_stats_stall(ReadStream *stream)
+{
+ stream->stats.stall_count++;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -743,7 +747,7 @@ read_stream_begin_impl(int flags,
stream->temporary = SmgrIsTemp(smgr);
/* zero the stats, then set capacity */
- memset(&stream->stats, 0, sizeof(ReadStreamInstrumentation));
+ memset(&stream->stats, 0, sizeof(IOStats));
stream->stats.distance_capacity = max_pinned_buffers;
/*
@@ -907,6 +911,9 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
/* update I/O stats */
read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_update_stats_prefetch(stream);
}
else
{
@@ -918,7 +925,6 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
stream->fast_path = false;
- read_stream_update_stats_prefetch(stream);
return buffer;
}
#endif
@@ -974,7 +980,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
/* Count it as a stall if we need to wait for IO */
if (needed_wait)
- stream->stats.stall_count += 1;
+ read_stream_update_stats_stall(stream);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -1175,7 +1181,7 @@ read_stream_end(ReadStream *stream)
}
/* return the prefetch stats for the read_stream */
-ReadStreamInstrumentation
+IOStats
read_stream_prefetch_stats(ReadStream *stream)
{
return stream->stats;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index cada9f28bd8..c0488987b2d 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -141,6 +141,20 @@ typedef struct TableScanStatsData
} TableScanStatsData;
typedef struct TableScanStatsData *TableScanStats;
+#define ACCUMULATE_TABLE_STATS(dst, src) \
+ do { \
+ (dst)->prefetch_count += (src)->prefetch_count; \
+ (dst)->distance_sum += (src)->distance_sum; \
+ if ((src)->distance_max > (dst)->distance_max) \
+ (dst)->distance_max = (src)->distance_max; \
+ if ((src)->distance_capacity > (dst)->distance_capacity) \
+ (dst)->distance_capacity = (src)->distance_capacity; \
+ (dst)->stall_count += (src)->stall_count; \
+ (dst)->io_count += (src)->io_count; \
+ (dst)->io_nblocks += (src)->io_nblocks; \
+ (dst)->io_in_progress += (src)->io_in_progress; \
+ } while (0)
+
/*
* Base class for fetches from a table via an index. This is the base-class
* for such scans, which needs to be embedded in the respective struct for
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index d94dbbc917d..cda00ad40e9 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -44,7 +44,7 @@ typedef struct SharedAggInfo
* Instrumentation information about read streams
* ---------------------
*/
-typedef struct ReadStreamInstrumentation
+typedef struct IOStats
{
/* number of buffers returned to consumer (for averaging distance) */
uint64 prefetch_count;
@@ -65,7 +65,7 @@ typedef struct ReadStreamInstrumentation
uint64 io_count; /* number of I/Os */
uint64 io_nblocks; /* sum of blocks for all I/Os */
uint64 io_in_progress; /* sum of in-progress I/Os */
-} ReadStreamInstrumentation;
+} IOStats;
/* ---------------------
@@ -74,7 +74,7 @@ typedef struct ReadStreamInstrumentation
*/
typedef struct SeqScanInstrumentation
{
- ReadStreamInstrumentation stream;
+ IOStats io;
} SeqScanInstrumentation;
/*
@@ -113,7 +113,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
- ReadStreamInstrumentation stream;
+ IOStats io;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index ec396db5369..0676bdc9b14 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,7 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
-typedef struct ReadStreamInstrumentation ReadStreamInstrumentation;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -105,6 +105,6 @@ extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
-extern ReadStreamInstrumentation read_stream_prefetch_stats(ReadStream *stream);
+extern IOStats read_stream_prefetch_stats(ReadStream *stream);
#endif /* READ_STREAM_H */
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..2d247f73c14 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -23,6 +23,10 @@ begin
-- Ignore text-mode buffers output because it varies depending
-- on the system state
CONTINUE WHEN (ln ~ ' +Buffers: .*');
+ -- Ignore text-mode prefetch and I/O output because it varies
+ -- depending on the system state
+ CONTINUE WHEN (ln ~ ' +Prefetch: .*');
+ CONTINUE WHEN (ln ~ ' +I/O: .*');
-- Ignore text-mode "Planning:" line because whether it's output
-- varies depending on the system state
CONTINUE WHEN (ln = 'Planning:');
@@ -68,7 +72,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +80,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -93,7 +97,7 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------
<explain xmlns="http://www.postgresql.org/N/explain"> +
@@ -144,7 +148,7 @@ select explain_filter('explain (analyze, buffers, format xml) select * from int8
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
explain_filter
-------------------------------
- Plan: +
@@ -310,6 +314,11 @@ select explain_filter('explain (analyze, buffers, format json) select * from int
"Actual Rows": N.N, +
"Actual Loops": N, +
"Disabled": false, +
+ "Prefetch": { +
+ "Average Distance": N.N, +
+ "Max Distance": N, +
+ "Capacity": N +
+ }, +
"Shared Hit Blocks": N, +
"Shared Read Blocks": N, +
"Shared Dirtied Blocks": N, +
@@ -396,7 +405,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -445,6 +454,11 @@ select explain_filter('explain (memory, analyze, format json) select * from int8
"Actual Rows": N.N, +
"Actual Loops": N, +
"Disabled": false, +
+ "Prefetch": { +
+ "Average Distance": N.N, +
+ "Max Distance": N, +
+ "Capacity": N +
+ }, +
"Shared Hit Blocks": N, +
"Shared Read Blocks": N, +
"Shared Dirtied Blocks": N, +
@@ -568,6 +582,11 @@ select jsonb_pretty(
], +
"Schema": "public", +
"Disabled": false, +
+ "Prefetch": { +
+ "Capacity": 0, +
+ "Max Distance": 0, +
+ "Average Distance": 0.0 +
+ }, +
"Node Type": "Seq Scan", +
"Plan Rows": 0, +
"Plan Width": 0, +
@@ -744,7 +763,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -772,7 +791,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -782,7 +801,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -795,7 +814,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -807,7 +826,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 00c30b91459..425aed4d749 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1332cd0e646 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
@@ -2322,7 +2322,8 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
Subplans Removed: 3
-> Seq Scan on list_part1 list_part_1 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(1))
-(4 rows)
+ Prefetch: avg=1.000 max=1 capacity=94
+(5 rows)
-- Ensure pruning does not take place when the function has a Var parameter
explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
@@ -2331,13 +2332,17 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
Append (actual rows=4.00 loops=1)
-> Seq Scan on list_part1 list_part_1 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part2 list_part_2 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part3 list_part_3 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part4 list_part_4 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
-(9 rows)
+ Prefetch: avg=1.000 max=1 capacity=94
+(13 rows)
-- Ensure pruning does not take place when the expression contains a Var.
explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
@@ -2347,16 +2352,20 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
-> Seq Scan on list_part1 list_part_1 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part2 list_part_2 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part3 list_part_3 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part4 list_part_4 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
-(13 rows)
+ Prefetch: avg=1.000 max=1 capacity=94
+(17 rows)
rollback;
drop table list_part;
@@ -2524,6 +2533,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Nested Loop (actual rows=N loops=N)
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{0,0,1}'::integer[]))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = a.a)
@@ -2543,7 +2553,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = a.a)
-(27 rows)
+(28 rows)
-- Ensure the same partitions are pruned when we make the nested loop
-- parameter an Expr rather than a plain Param.
@@ -2558,6 +2568,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Nested Loop (actual rows=N loops=N)
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{0,0,1}'::integer[]))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = (a.a + 0))
@@ -2577,7 +2588,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = (a.a + 0))
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = (a.a + 0))
-(27 rows)
+(28 rows)
insert into lprt_a values(3),(3);
select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on ab.a = a.a where a.a in(1, 0, 3)');
@@ -2591,6 +2602,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Nested Loop (actual rows=N loops=N)
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{1,0,3}'::integer[]))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = a.a)
@@ -2610,7 +2622,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (actual rows=N loops=N)
Index Cond: (a = a.a)
-(27 rows)
+(28 rows)
select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on ab.a = a.a where a.a in(1, 0, 0)');
explain_parallel_append
@@ -2624,6 +2636,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{1,0,0}'::integer[]))
Rows Removed by Filter: N
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = a.a)
@@ -2643,7 +2656,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = a.a)
-(28 rows)
+(29 rows)
delete from lprt_a where a = 1;
select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on ab.a = a.a where a.a in(1, 0, 0)');
@@ -2658,6 +2671,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{1,0,0}'::integer[]))
Rows Removed by Filter: N
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (never executed)
Index Cond: (a = a.a)
@@ -2677,7 +2691,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = a.a)
-(28 rows)
+(29 rows)
reset enable_hashjoin;
reset enable_mergejoin;
@@ -2695,9 +2709,11 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
InitPlan expr_1
-> Aggregate (actual rows=1.00 loops=1)
-> Seq Scan on lprt_a (actual rows=102.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
InitPlan expr_2
-> Aggregate (actual rows=1.00 loops=1)
-> Seq Scan on lprt_a lprt_a_1 (actual rows=102.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Bitmap Heap Scan on ab_a1_b1 ab_1 (never executed)
Recheck Cond: (a = (InitPlan expr_1).col1)
Filter: (b = (InitPlan expr_2).col1)
@@ -2752,7 +2768,7 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
-> Bitmap Index Scan on ab_a3_b3_a_idx (never executed)
Index Cond: (a = (InitPlan expr_1).col1)
Index Searches: 0
-(61 rows)
+(63 rows)
-- Test run-time partition pruning with UNION ALL parents
explain (analyze, costs off, summary off, timing off, buffers off)
@@ -2880,13 +2896,14 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6
-> Seq Scan on xy_1 (actual rows=0.00 loops=1)
Filter: ((x = $1) AND (y = (InitPlan expr_1).col1))
Rows Removed by Filter: 1
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on ab_a1_b1 ab_4 (never executed)
Filter: ((a = $1) AND (b = (InitPlan expr_1).col1))
-> Seq Scan on ab_a1_b2 ab_5 (never executed)
Filter: ((a = $1) AND (b = (InitPlan expr_1).col1))
-> Seq Scan on ab_a1_b3 ab_6 (never executed)
Filter: ((a = $1) AND (b = (InitPlan expr_1).col1))
-(19 rows)
+(20 rows)
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -3025,6 +3042,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-----------------------------------------------------------------------------
Nested Loop (actual rows=6.00 loops=1)
-> Seq Scan on tbl1 (actual rows=2.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=3.00 loops=2)
-> Index Scan using tprt1_idx on tprt_1 (actual rows=2.00 loops=2)
Index Cond: (col1 < tbl1.col1)
@@ -3044,7 +3062,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 < tbl1.col1)
Index Searches: 0
-(21 rows)
+(22 rows)
explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
@@ -3052,6 +3070,7 @@ select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
-----------------------------------------------------------------------------
Nested Loop (actual rows=2.00 loops=1)
-> Seq Scan on tbl1 (actual rows=2.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=1.00 loops=2)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 = tbl1.col1)
@@ -3071,7 +3090,7 @@ select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 = tbl1.col1)
Index Searches: 0
-(21 rows)
+(22 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 > tprt.col1
@@ -3103,6 +3122,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-----------------------------------------------------------------------------
Nested Loop (actual rows=23.00 loops=1)
-> Seq Scan on tbl1 (actual rows=5.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=4.60 loops=5)
-> Index Scan using tprt1_idx on tprt_1 (actual rows=2.00 loops=5)
Index Cond: (col1 < tbl1.col1)
@@ -3122,7 +3142,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 < tbl1.col1)
Index Searches: 0
-(21 rows)
+(22 rows)
explain (analyze, costs off, summary off, timing off, buffers off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
@@ -3130,6 +3150,7 @@ select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
-----------------------------------------------------------------------------
Nested Loop (actual rows=3.00 loops=1)
-> Seq Scan on tbl1 (actual rows=5.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=0.60 loops=5)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 = tbl1.col1)
@@ -3149,7 +3170,7 @@ select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 = tbl1.col1)
Index Searches: 0
-(21 rows)
+(22 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 > tprt.col1
@@ -3200,6 +3221,7 @@ select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
-----------------------------------------------------------------------------
Nested Loop (actual rows=1.00 loops=1)
-> Seq Scan on tbl1 (actual rows=1.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=1.00 loops=1)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 > tbl1.col1)
@@ -3219,7 +3241,7 @@ select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (actual rows=1.00 loops=1)
Index Cond: (col1 > tbl1.col1)
Index Searches: 1
-(21 rows)
+(22 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 < tprt.col1
@@ -3238,6 +3260,7 @@ select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
-------------------------------------------------------------------
Nested Loop (actual rows=0.00 loops=1)
-> Seq Scan on tbl1 (actual rows=1.00 loops=1)
+ Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=0.00 loops=1)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 = tbl1.col1)
@@ -3257,7 +3280,7 @@ select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 = tbl1.col1)
Index Searches: 0
-(21 rows)
+(22 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 = tprt.col1
@@ -3485,11 +3508,14 @@ select * from mc3p where a < 3 and abs(b) = 1;
Append (actual rows=3.00 loops=1)
-> Seq Scan on mc3p0 mc3p_1 (actual rows=1.00 loops=1)
Filter: ((a < 3) AND (abs(b) = 1))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on mc3p1 mc3p_2 (actual rows=1.00 loops=1)
Filter: ((a < 3) AND (abs(b) = 1))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on mc3p2 mc3p_3 (actual rows=1.00 loops=1)
Filter: ((a < 3) AND (abs(b) = 1))
-(7 rows)
+ Prefetch: avg=1.000 max=1 capacity=94
+(10 rows)
--
-- Check that pruning with composite range partitioning works correctly when
@@ -3508,7 +3534,8 @@ execute ps1(1);
-> Result (actual rows=1.00 loops=1)
-> Seq Scan on mc3p1 mc3p_1 (actual rows=1.00 loops=1)
Filter: ((a = $1) AND (abs(b) < (InitPlan expr_1).col1))
-(6 rows)
+ Prefetch: avg=1.000 max=1 capacity=94
+(7 rows)
deallocate ps1;
prepare ps2 as
@@ -3523,9 +3550,11 @@ execute ps2(1);
-> Result (actual rows=1.00 loops=1)
-> Seq Scan on mc3p0 mc3p_1 (actual rows=1.00 loops=1)
Filter: ((a <= $1) AND (abs(b) < (InitPlan expr_1).col1))
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on mc3p1 mc3p_2 (actual rows=1.00 loops=1)
Filter: ((a <= $1) AND (abs(b) < (InitPlan expr_1).col1))
-(8 rows)
+ Prefetch: avg=1.000 max=1 capacity=94
+(10 rows)
deallocate ps2;
drop table mc3p;
@@ -3544,11 +3573,12 @@ select * from boolp where a = (select value from boolvalues where value);
-> Seq Scan on boolvalues (actual rows=1.00 loops=1)
Filter: value
Rows Removed by Filter: 1
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on boolp_f boolp_1 (never executed)
Filter: (a = (InitPlan expr_1).col1)
-> Seq Scan on boolp_t boolp_2 (actual rows=0.00 loops=1)
Filter: (a = (InitPlan expr_1).col1)
-(9 rows)
+(10 rows)
explain (analyze, costs off, summary off, timing off, buffers off)
select * from boolp where a = (select value from boolvalues where not value);
@@ -3559,11 +3589,12 @@ select * from boolp where a = (select value from boolvalues where not value);
-> Seq Scan on boolvalues (actual rows=1.00 loops=1)
Filter: (NOT value)
Rows Removed by Filter: 1
+ Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on boolp_f boolp_1 (actual rows=0.00 loops=1)
Filter: (a = (InitPlan expr_1).col1)
-> Seq Scan on boolp_t boolp_2 (never executed)
Filter: (a = (InitPlan expr_1).col1)
-(9 rows)
+(10 rows)
drop table boolp;
--
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..e9a3c2caf91 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -25,6 +25,10 @@ begin
-- Ignore text-mode buffers output because it varies depending
-- on the system state
CONTINUE WHEN (ln ~ ' +Buffers: .*');
+ -- Ignore text-mode prefetch and I/O output because it varies
+ -- depending on the system state
+ CONTINUE WHEN (ln ~ ' +Prefetch: .*');
+ CONTINUE WHEN (ln ~ ' +I/O: .*');
-- Ignore text-mode "Planning:" line because whether it's output
-- varies depending on the system state
CONTINUE WHEN (ln = 'Planning:');
@@ -63,11 +67,11 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
@@ -102,7 +106,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -174,17 +178,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index 8d1cdd6990c..d1f80e18b41 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
--
2.53.0
[text/x-patch] v3-0004-store-the-IO-stats-in-scan-descriptor.patch (24.3K, ../../bf1d4799-e2fd-464c-96ff-dc448f021b1e@vondra.me/5-v3-0004-store-the-IO-stats-in-scan-descriptor.patch)
download | inline diff:
From fec8ab8c5bac3ed43cec1ccbbbb53e0fafb541eb Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 16:03:47 +0100
Subject: [PATCH v3 4/6] store the IO stats in scan descriptor
- Don't pass the stats through the TAM interface. Instead, make them
part of the scan descriptor (optionally).
- ambeginscan() may allocate IOStats in the scan descriptor, and fill it
with data. Being part of the decriptor means it won't survive
amendscan().
- The IOStats is then passed to the ReadStream through a new argument
added to read_stream_begin_relation(), and the stream updates this
place directly. There's no need to "fetch" the stats from the stream.
- With parallel query, the data from workers is still copied through
shared memory by endscan() in the workers. The leader has to consume
and merge the data. The only difference is that the workers read the
data from the scan descriptor, and not from the instrumentation.
---
contrib/amcheck/verify_heapam.c | 3 +-
contrib/bloom/blscan.c | 3 +-
contrib/bloom/blvacuum.c | 6 ++-
contrib/pg_prewarm/autoprewarm.c | 3 +-
contrib/pg_prewarm/pg_prewarm.c | 3 +-
contrib/pg_visibility/pg_visibility.c | 6 ++-
contrib/pgstattuple/pgstatapprox.c | 3 +-
contrib/pgstattuple/pgstatindex.c | 6 ++-
src/backend/access/brin/brin.c | 3 +-
src/backend/access/gin/ginvacuum.c | 3 +-
src/backend/access/gist/gistvacuum.c | 3 +-
src/backend/access/hash/hash.c | 3 +-
src/backend/access/heap/heapam.c | 41 +++-------------
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/access/heap/vacuumlazy.c | 6 ++-
src/backend/access/nbtree/nbtree.c | 3 +-
src/backend/access/spgist/spgvacuum.c | 3 +-
src/backend/commands/analyze.c | 3 +-
src/backend/commands/explain.c | 31 ++----------
src/backend/executor/nodeBitmapHeapscan.c | 6 +--
src/backend/executor/nodeSeqscan.c | 6 +--
src/backend/storage/aio/read_stream.c | 59 ++++++++++++++---------
src/include/access/heapam.h | 1 -
src/include/access/relscan.h | 6 +++
src/include/access/tableam.h | 17 -------
src/include/nodes/execnodes.h | 1 -
src/include/storage/read_stream.h | 5 +-
27 files changed, 97 insertions(+), 137 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index ada782f98f5..0ec4dc849c3 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -476,7 +476,8 @@ verify_heapam(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
stream_cb,
stream_data,
- 0);
+ 0,
+ NULL);
while ((ctx.buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
{
diff --git a/contrib/bloom/blscan.c b/contrib/bloom/blscan.c
index 1a0e42021ec..9bdc4a95c18 100644
--- a/contrib/bloom/blscan.c
+++ b/contrib/bloom/blscan.c
@@ -138,7 +138,8 @@ blgetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
index 6beb1c20ebb..587757c1e18 100644
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -71,7 +71,8 @@ blbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
@@ -224,7 +225,8 @@ blvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index ba0bc8e6d4a..4c5448f177a 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -628,7 +628,8 @@ autoprewarm_database_main(Datum main_arg)
p.forknum,
apw_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/*
* Loop until we've prewarmed all the blocks from this fork. The
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index c2716086693..e4541f46777 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -256,7 +256,8 @@ pg_prewarm(PG_FUNCTION_ARGS)
forkNumber,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (block = first_block; block <= last_block; ++block)
{
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dfab0b64cf5..f09bb68624f 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -519,7 +519,8 @@ collect_visibility_data(Oid relid, bool include_pd)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
}
for (blkno = 0; blkno < nblocks; ++blkno)
@@ -739,7 +740,8 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen)
MAIN_FORKNUM,
collect_corrupt_items_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/* Loop over every block in the relation. */
while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 21e0b50fb4b..a926a1f1168 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -146,7 +146,8 @@ statapprox_heap(Relation rel, output_type *stat)
MAIN_FORKNUM,
statapprox_heap_read_stream_next,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 3a3f2637bd9..0de33558c47 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -296,7 +296,8 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
@@ -685,7 +686,8 @@ pgstathashindex(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 2a0f8c8e3b8..eb4e4c5de4a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2192,7 +2192,8 @@ brin_vacuum_scan(Relation idxrel, BufferAccessStrategy strategy)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* Scan the index in physical order, and clean up any possible mess in
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 840543eb664..484360a9d6d 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -799,7 +799,8 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = GIN_ROOT_BLKNO; blkno < npages; blkno++)
{
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 686a0418054..e4f578b2de0 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -223,7 +223,8 @@ gistvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 6df5e7ccbd1..043088dbee8 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -546,7 +546,8 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
hash_bulkdelete_read_stream_cb,
&stream_private,
- 0);
+ 0,
+ NULL);
bucket_loop:
while (cur_bucket <= cur_maxbucket)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 323badd27f1..6acbcf1f342 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1267,6 +1267,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_read_stream = NULL;
+ /* reset the stats counters */
+ memset(&scan->rs_base.rs_iostats, 0, sizeof(IOStats));
+
/*
* Set up a read stream for sequential scans and TID range scans. This
* should be done after initscan() because initscan() allocates the
@@ -1296,7 +1299,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
cb,
scan,
- 0);
+ 0,
+ &scan->rs_base.rs_iostats);
}
else if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
{
@@ -1307,7 +1311,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
bitmapheap_stream_read_next,
scan,
- sizeof(TBMIterateResult));
+ sizeof(TBMIterateResult),
+ &scan->rs_base.rs_iostats);
}
scan->rs_vmbuffer = InvalidBuffer;
@@ -1416,38 +1421,6 @@ heap_endscan(TableScanDesc sscan)
pfree(scan);
}
-/*
- * heap_scan_stats
- * return stats collected by the read stream
- *
- * Returns NULL if the scan is not using a read stream.
- */
-TableScanStats
-heap_scan_stats(TableScanDesc sscan)
-{
- HeapScanDesc scan = (HeapScanDesc) sscan;
- IOStats stats;
- TableScanStats res;
-
- if (!scan->rs_read_stream)
- return NULL;
-
- stats = read_stream_prefetch_stats(scan->rs_read_stream);
-
- res = palloc0(sizeof(TableScanStatsData));
-
- res->prefetch_count = stats.prefetch_count;
- res->distance_sum = stats.distance_sum;
- res->distance_max = stats.distance_max;
- res->distance_capacity = stats.distance_capacity;
- res->stall_count = stats.stall_count;
- res->io_count = stats.io_count;
- res->io_nblocks = stats.io_nblocks;
- res->io_in_progress = stats.io_in_progress;
-
- return res;
-}
-
HeapTuple
heap_getnext(TableScanDesc sscan, ScanDirection direction)
{
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 51ac2164afd..253a735b6c1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2650,7 +2650,6 @@ static const TableAmRoutine heapam_methods = {
.scan_set_tidrange = heap_set_tidrange,
.scan_getnextslot_tidrange = heap_getnextslot_tidrange,
- .scan_stats = heap_scan_stats,
.parallelscan_estimate = table_block_parallelscan_estimate,
.parallelscan_initialize = table_block_parallelscan_initialize,
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 82c5b28e0ad..bec454de6d8 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1290,7 +1290,8 @@ lazy_scan_heap(LVRelState *vacrel)
MAIN_FORKNUM,
heap_vac_scan_next_block,
vacrel,
- sizeof(bool));
+ sizeof(bool),
+ NULL);
while (true)
{
@@ -2790,7 +2791,8 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
MAIN_FORKNUM,
vacuum_reap_lp_read_stream_next,
iter,
- sizeof(TidStoreIterResult));
+ sizeof(TidStoreIterResult),
+ NULL);
while (true)
{
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index aed74590cf4..e5cd51cce4d 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -1325,7 +1325,8 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index 6b7117b56b2..206a8e9c8b0 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -835,7 +835,8 @@ spgvacuumscan(spgBulkDeleteState *bds)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* The outer loop iterates over all index pages except the metapage, in
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index eeed91be266..a2cf176a485 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -1251,7 +1251,8 @@ acquire_sample_rows(Relation onerel, int elevel,
MAIN_FORKNUM,
block_sampling_read_stream_next,
&bs,
- 0);
+ 0,
+ NULL);
/* Outer loop over blocks to sample */
while (table_scan_analyze_next_block(scan, stream))
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 7a4009ae74f..25af6550098 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3998,7 +3998,7 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
static void
-print_io_usage(ExplainState *es, TableScanStats stats)
+print_io_usage(ExplainState *es, IOStats *stats)
{
/* don't print stats if there's nothing to report */
if (stats->prefetch_count > 0)
@@ -4067,8 +4067,7 @@ static void
show_scan_io_usage(ScanState *planstate, ExplainState *es)
{
Plan *plan = planstate->ps.plan;
- TableScanStats leader_stats;
- TableScanStatsData stats;
+ IOStats stats;
if (!es->io)
return;
@@ -4078,16 +4077,7 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
return;
/* collect prefetch statistics from the read stream */
- leader_stats = table_scan_stats(planstate->ss_currentScanDesc);
-
- if (leader_stats)
- {
- memcpy(&stats, leader_stats, sizeof(TableScanStatsData));
- }
- else
- {
- memset(&stats, 0, sizeof(TableScanStatsData));
- }
+ stats = planstate->ss_currentScanDesc->rs_iostats;
/* Initialize counters with stats from the local process first */
switch (nodeTag(plan))
@@ -4146,9 +4136,6 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
Plan *plan = planstate->plan;
IOStats *stats = NULL;
- // XXX
- TableScanStatsData tmp;
-
if (!es->io)
return;
@@ -4180,17 +4167,7 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
return;
}
- /* XXX convert to TableScanStats */
- tmp.distance_capacity = stats->distance_capacity;
- tmp.distance_max = stats->distance_max;
- tmp.distance_sum = stats->distance_sum;
- tmp.io_count = stats->io_count;
- tmp.io_in_progress = stats->io_in_progress;
- tmp.io_nblocks = stats->io_nblocks;
- tmp.prefetch_count = stats->prefetch_count;
- tmp.stall_count = stats->stall_count;
-
- print_io_usage(es, &tmp);
+ print_io_usage(es, stats);
}
/*
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 0f1459f11a4..55b1e294d6c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -315,7 +315,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
if (node->sinstrument != NULL && IsParallelWorker())
{
BitmapHeapScanInstrumentation *si;
- TableScanStats stats;
Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
@@ -331,10 +330,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
si->lossy_pages += node->stats.lossy_pages;
/* collect prefetch info for this process from the read_stream */
- if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
- {
- ACCUMULATE_TABLE_STATS(stats, &si->io);
- }
+ ACCUMULATE_TABLE_STATS(&node->ss.ss_currentScanDesc->rs_iostats, &si->io);
}
/*
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 4fa85a1b6bc..28e33aa391d 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -303,7 +303,6 @@ ExecEndSeqScan(SeqScanState *node)
if (node->sinstrument != NULL && IsParallelWorker())
{
SeqScanInstrumentation *si;
- TableScanStats stats;
Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
@@ -317,10 +316,7 @@ ExecEndSeqScan(SeqScanState *node)
*/
/* collect prefetch info for this process from the read_stream */
- if ((stats = table_scan_stats(node->ss.ss_currentScanDesc)) != NULL)
- {
- ACCUMULATE_TABLE_STATS(stats, &si->io);
- }
+ ACCUMULATE_TABLE_STATS(&node->ss.ss_currentScanDesc->rs_iostats, &si->io);
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index faa52ca4b2c..56b0b8e7fb5 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -109,7 +109,7 @@ struct ReadStream
bool temporary;
/* scan stats counters */
- IOStats stats;
+ IOStats *stats;
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
@@ -186,10 +186,15 @@ block_range_read_stream_cb(ReadStream *stream,
static inline void
read_stream_update_stats_prefetch(ReadStream *stream)
{
- stream->stats.prefetch_count++;
- stream->stats.distance_sum += stream->pinned_buffers;
- if (stream->pinned_buffers > stream->stats.distance_max)
- stream->stats.distance_max = stream->pinned_buffers;
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
}
/*
@@ -201,15 +206,25 @@ read_stream_update_stats_prefetch(ReadStream *stream)
static inline void
read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
{
- stream->stats.io_count++;
- stream->stats.io_nblocks += nblocks;
- stream->stats.io_in_progress += in_progress;
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
}
static inline void
read_stream_update_stats_stall(ReadStream *stream)
{
- stream->stats.stall_count++;
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->stall_count++;
}
/*
@@ -587,7 +602,8 @@ read_stream_begin_impl(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
ReadStream *stream;
size_t size;
@@ -746,9 +762,10 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
- /* zero the stats, then set capacity */
- memset(&stream->stats, 0, sizeof(IOStats));
- stream->stats.distance_capacity = max_pinned_buffers;
+ /* set capacity */
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = max_pinned_buffers;
/*
* Skip the initial ramp-up phase if the caller says we're going to be
@@ -789,7 +806,8 @@ read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
return read_stream_begin_impl(flags,
strategy,
@@ -799,7 +817,8 @@ read_stream_begin_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ stats);
}
/*
@@ -824,7 +843,8 @@ read_stream_begin_smgr_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ NULL);
}
/*
@@ -1179,10 +1199,3 @@ read_stream_end(ReadStream *stream)
read_stream_reset(stream);
pfree(stream);
}
-
-/* return the prefetch stats for the read_stream */
-IOStats
-read_stream_prefetch_stats(ReadStream *stream)
-{
- return stream->stats;
-}
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 81dc0b7521d..2fdc50b865b 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -359,7 +359,6 @@ extern void heap_prepare_pagescan(TableScanDesc sscan);
extern void heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
bool allow_strat, bool allow_sync, bool allow_pagemode);
extern void heap_endscan(TableScanDesc sscan);
-extern TableScanStats heap_scan_stats(TableScanDesc sscan);
extern HeapTuple heap_getnext(TableScanDesc sscan, ScanDirection direction);
extern bool heap_getnextslot(TableScanDesc sscan,
ScanDirection direction, TupleTableSlot *slot);
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index c0488987b2d..e41f0f8ee13 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -16,6 +16,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
+#include "executor/instrument_node.h"
#include "nodes/tidbitmap.h"
#include "port/atomics.h"
#include "storage/relfilelocator.h"
@@ -62,6 +63,11 @@ typedef struct TableScanDescData
*/
uint32 rs_flags;
+ /*
+ * IO statistics for the scan, optionally filled by the table AM.
+ */
+ IOStats rs_iostats;
+
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
} TableScanDescData;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ab84b76443c..06084752245 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -380,11 +380,6 @@ typedef struct TableAmRoutine
ScanDirection direction,
TupleTableSlot *slot);
- /*
- * Collect statistics about table scan.
- */
- TableScanStats (*scan_stats) (TableScanDesc scan);
-
/* ------------------------------------------------------------------------
* Parallel table scan related functions.
* ------------------------------------------------------------------------
@@ -1011,18 +1006,6 @@ table_endscan(TableScanDesc scan)
scan->rs_rd->rd_tableam->scan_end(scan);
}
-/*
- * Fetch statistics about table scan.
- */
-static inline TableScanStats
-table_scan_stats(TableScanDesc scan)
-{
- if (scan->rs_rd->rd_tableam->scan_stats)
- return scan->rs_rd->rd_tableam->scan_stats(scan);
-
- return NULL;
-}
-
/*
* Restart a relation scan.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 96a8a75031f..f287f8fc813 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1644,7 +1644,6 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
- SeqScanInstrumentation stats;
SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index 0676bdc9b14..b56bf925fc1 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -88,7 +88,8 @@ extern ReadStream *read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size);
+ size_t per_buffer_data_size,
+ IOStats *stat);
extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_data);
extern BlockNumber read_stream_next_block(ReadStream *stream,
BufferAccessStrategy *strategy);
@@ -105,6 +106,4 @@ extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
-extern IOStats read_stream_prefetch_stats(ReadStream *stream);
-
#endif /* READ_STREAM_H */
--
2.53.0
[text/x-patch] v3-0005-get-rid-of-TableScanStats-Data.patch (5.3K, ../../bf1d4799-e2fd-464c-96ff-dc448f021b1e@vondra.me/6-v3-0005-get-rid-of-TableScanStats-Data.patch)
download | inline diff:
From 940dad96d9810692e660b0f289af7c7ed2bf2022 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 16:30:28 +0100
Subject: [PATCH v3 5/6] get rid of TableScanStats[Data]
finish replacing it with IOStats
---
src/backend/commands/explain.c | 4 +--
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/backend/executor/nodeSeqscan.c | 2 +-
src/include/access/relscan.h | 41 -----------------------
src/include/executor/instrument_node.h | 13 +++++++
5 files changed, 17 insertions(+), 45 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 25af6550098..1b0feeadd9e 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4093,7 +4093,7 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
for (int i = 0; i < sinstrument->num_workers; ++i)
{
SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
- ACCUMULATE_TABLE_STATS(&stats, &winstrument->io);
+ ACCUMULATE_IO_STATS(&stats, &winstrument->io);
}
}
@@ -4110,7 +4110,7 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
for (int i = 0; i < sinstrument->num_workers; ++i)
{
BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
- ACCUMULATE_TABLE_STATS(&stats, &winstrument->io);
+ ACCUMULATE_IO_STATS(&stats, &winstrument->io);
}
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 55b1e294d6c..b785cb7a237 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -330,7 +330,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
si->lossy_pages += node->stats.lossy_pages;
/* collect prefetch info for this process from the read_stream */
- ACCUMULATE_TABLE_STATS(&node->ss.ss_currentScanDesc->rs_iostats, &si->io);
+ ACCUMULATE_IO_STATS(&si->io, &node->ss.ss_currentScanDesc->rs_iostats);
}
/*
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 28e33aa391d..2ecc0e3054d 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -316,7 +316,7 @@ ExecEndSeqScan(SeqScanState *node)
*/
/* collect prefetch info for this process from the read_stream */
- ACCUMULATE_TABLE_STATS(&node->ss.ss_currentScanDesc->rs_iostats, &si->io);
+ ACCUMULATE_IO_STATS(&si->io, &node->ss.ss_currentScanDesc->rs_iostats);
}
/*
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index e41f0f8ee13..32ef27c546f 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -120,47 +120,6 @@ typedef struct ParallelBlockTableScanWorkerData
} ParallelBlockTableScanWorkerData;
typedef struct ParallelBlockTableScanWorkerData *ParallelBlockTableScanWorker;
-/*
- * Generic prefetch stats for table scans.
- */
-typedef struct TableScanStatsData
-{
- /* number of buffers returned to consumer (for averaging distance) */
- uint64 prefetch_count;
-
- /* sum of pinned_buffers sampled at each buffer return */
- uint64 distance_sum;
-
- /* maximum actual pinned_buffers observed during the scan */
- int16 distance_max;
-
- /* maximum possible look-ahead distance (max_pinned_buffers) */
- int16 distance_capacity;
-
- /* number of stalled reads (waiting for I/O) */
- uint64 stall_count;
-
- /* I/O stats */
- uint64 io_count; /* number of I/Os */
- uint64 io_nblocks; /* sum of blocks for all I/Os */
- uint64 io_in_progress; /* sum of in-progress I/Os */
-} TableScanStatsData;
-typedef struct TableScanStatsData *TableScanStats;
-
-#define ACCUMULATE_TABLE_STATS(dst, src) \
- do { \
- (dst)->prefetch_count += (src)->prefetch_count; \
- (dst)->distance_sum += (src)->distance_sum; \
- if ((src)->distance_max > (dst)->distance_max) \
- (dst)->distance_max = (src)->distance_max; \
- if ((src)->distance_capacity > (dst)->distance_capacity) \
- (dst)->distance_capacity = (src)->distance_capacity; \
- (dst)->stall_count += (src)->stall_count; \
- (dst)->io_count += (src)->io_count; \
- (dst)->io_nblocks += (src)->io_nblocks; \
- (dst)->io_in_progress += (src)->io_in_progress; \
- } while (0)
-
/*
* Base class for fetches from a table via an index. This is the base-class
* for such scans, which needs to be embedded in the respective struct for
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index cda00ad40e9..ce643ef0635 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -67,6 +67,19 @@ typedef struct IOStats
uint64 io_in_progress; /* sum of in-progress I/Os */
} IOStats;
+#define ACCUMULATE_IO_STATS(dst, src) \
+ do { \
+ (dst)->prefetch_count += (src)->prefetch_count; \
+ (dst)->distance_sum += (src)->distance_sum; \
+ if ((src)->distance_max > (dst)->distance_max) \
+ (dst)->distance_max = (src)->distance_max; \
+ if ((src)->distance_capacity > (dst)->distance_capacity) \
+ (dst)->distance_capacity = (src)->distance_capacity; \
+ (dst)->stall_count += (src)->stall_count; \
+ (dst)->io_count += (src)->io_count; \
+ (dst)->io_nblocks += (src)->io_nblocks; \
+ (dst)->io_in_progress += (src)->io_in_progress; \
+ } while (0)
/* ---------------------
* Instrumentation information for sequential scans
--
2.53.0
[text/x-patch] v3-0006-show-prefetch-stats-for-TidRangeScan.patch (9.9K, ../../bf1d4799-e2fd-464c-96ff-dc448f021b1e@vondra.me/7-v3-0006-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From 8566941dfeb29f06880984bcd99db04ac3029a82 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 20:55:00 +0100
Subject: [PATCH v3 6/6] show prefetch stats for TidRangeScan
- enable show_scan_io_usage for TidRangeScan
- add infrastructure to allocate/collect instrumentation from parallel
workers
---
src/backend/commands/explain.c | 28 ++++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeTidrangescan.c | 110 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 11 +++
src/include/executor/nodeTidrangescan.h | 1 +
src/include/nodes/execnodes.h | 1 +
6 files changed, 149 insertions(+), 5 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 1b0feeadd9e..ee32fbafa9d 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2152,6 +2152,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4114,6 +4115,23 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->io);
+ }
+ }
+
break;
}
default:
@@ -4160,6 +4178,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
stats = &instrument->io;
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ TidRangeScanState *state = ((TidRangeScanState *) planstate);
+ SharedTidRangeScanInstrumentation *sinstrument = state->trss_sinstrument;
+ TidRangeScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->io;
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ffc708ed6be..62b26bfd18a 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1122,6 +1122,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 617713bde04..4ff6e333144 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -340,6 +340,29 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new state and zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->io, &node->ss.ss_currentScanDesc->rs_iostats);
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -433,11 +456,23 @@ void
ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->trss_pscanlen = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
- node->trss_pscanlen =
- table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->trss_pscanlen);
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -452,8 +487,18 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
- pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
+ size = MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
@@ -461,6 +506,23 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+ }
+
+ node->trss_sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -489,10 +551,48 @@ void
ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
pscan);
+
+ /*
+ * Workers don't get the trss_pscanlen value in scan descriptor, so use
+ * the TAM callback again. The result has to match the earlier result in
+ * ExecTidRangeScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecTidRangeScanRetrieveInstrumentation
+ *
+ * Transfer scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index ce643ef0635..b33b296ce03 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -99,6 +99,17 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+typedef struct TidRangeScanInstrumentation
+{
+ IOStats io;
+} TidRangeScanInstrumentation;
+
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..4f5dd74e0fc 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -27,5 +27,6 @@ extern void ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *p
extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f287f8fc813..611a6280987 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1896,6 +1896,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-19 18:11 ` Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Lukas Fittl @ 2026-03-19 18:11 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Mar 18, 2026 at 3:41 PM Tomas Vondra <tomas@vondra.me> wrote:
> The 0003 also changes the EXPLAIN to enable IO by default, just like we
> do for BUFFERS. It seems like a reasonable precedent to me.
One side effect of that is that the tests now fail for me locally,
because the specific values are system-dependent. Attached a patch
(nocfbot-0002) that fixed that for me.
There is one detail maybe calling out specifically on JSON output:
Currently Postgres always emits all fields in JSON output, even if
they are zero. The code that you have in v3 skips the "I/O" group when
the value is zero, which doesn't work well with how current regression
tests are written. I'm definitely not a fan of the unnecessary
verbosity of JSON EXPLAIN output, but I'd suggest we don't break with
the tradition here, and instead always output the "I/O" group in
non-text formats. Also attached a patch for that (nocfbot-0001).
Overall I think the abstraction here seems reasonable if we're
primarily focused on getting the per-node instrumentation taken care
of.
That said, two thoughts on an example EXPLAIN output I just ran:
1) I do wonder if its a bit confusing that we propagate I/O timings up
the EXPLAIN tree, but not the "I/O" information - I realize fixing
that would be a bit involved though, e.g. we'd have to invent
accumulation logic in explain.c. It'd also maybe make people thing
this covers things like temporary file reads/etc.
2) The ordering of "I/O Timings" in relation to "I/O" feels off to me
(since they're not next to each other) - maybe we should re-order I/O
Timings to come before Buffers in show_buffer_usage to address that?
EXPLAIN (ANALYZE) SELECT COUNT(*) FROM t;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------
Aggregate (cost=218331.00..218331.01 rows=1 width=8) (actual
time=563.437..563.437 rows=1.00 loops=1)
Buffers: shared hit=15806 read=41274
I/O Timings: shared read=1.180
-> Seq Scan on t (cost=0.00..186080.80 rows=12900080 width=0)
(actual time=0.335..306.737 rows=12900005.00 loops=1)
Prefetch: avg=61.517 max=91 capacity=94
I/O: stalls=7 size=14.825 inprogress=5.321
Buffers: shared hit=15806 read=41274
I/O Timings: shared read=1.180
Planning Time: 0.101 ms
Execution Time: 563.471 ms
(10 rows)
Thanks,
Lukas
--
Lukas Fittl
Attachments:
[application/octet-stream] nocfbot-0001-Emit-I-O-group-always-for-non-text-formats.patch (1.6K, ../../CAP53Pkzd9P=k_DsxU4rGoCEO-26rr9NQcS8co2y4HXB7TyiMbA@mail.gmail.com/2-nocfbot-0001-Emit-I-O-group-always-for-non-text-formats.patch)
download | inline diff:
From 24785402b25337a30a91ce95889f1dd61e4b84c7 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 19 Mar 2026 08:31:34 -0700
Subject: [PATCH 1/2] Emit I/O group always for non-text formats
---
src/backend/commands/explain.c | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ee32fbafa9d..28da983ad29 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4041,19 +4041,16 @@ print_io_usage(ExplainState *es, IOStats *stats)
ExplainCloseGroup("Prefetch", "Prefetch", true, es);
- if (stats->io_count > 0)
- {
- ExplainOpenGroup("I/O", "I/O", true, es);
+ ExplainOpenGroup("I/O", "I/O", true, es);
- ExplainPropertyUInteger("Stalls", NULL,
- stats->stall_count, es);
- ExplainPropertyFloat("Average IO Size", NULL,
- (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
- ExplainPropertyFloat("Average IOs In Progress", NULL,
- (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+ ExplainPropertyFloat("Average IO Size", NULL,
+ stats->io_count > 0 ? (stats->io_nblocks * 1.0 / stats->io_count) : 0, 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ stats->io_count > 0 ? (stats->io_in_progress * 1.0 / stats->io_count) : 0, 3, es);
- ExplainCloseGroup("I/O", "I/O", true, es);
- }
+ ExplainCloseGroup("I/O", "I/O", true, es);
}
}
}
--
2.47.1
[application/octet-stream] nocfbot-0002-Fix-regression-test-failures-due-to-defaulting-to-IO.patch (108.6K, ../../CAP53Pkzd9P=k_DsxU4rGoCEO-26rr9NQcS8co2y4HXB7TyiMbA@mail.gmail.com/3-nocfbot-0002-Fix-regression-test-failures-due-to-defaulting-to-IO.patch)
download | inline diff:
From 42c1160912a56593f0e0ceb1a4b5f91a77e38fc7 Mon Sep 17 00:00:00 2001
From: Lukas Fittl <lukas@fittl.com>
Date: Thu, 19 Mar 2026 11:09:39 -0700
Subject: [PATCH 2/2] Fix regression test failures due to defaulting to IO ON
for EXPLAIN
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/test/isolation/expected/merge-update.out | 6 +-
src/test/isolation/specs/merge-update.spec | 4 +-
.../modules/index/expected/killtuples.out | 44 +-
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/regress/expected/explain.out | 585 +++++++++---------
src/test/regress/expected/partition_prune.out | 163 ++---
src/test/regress/sql/partition_prune.sql | 90 +--
9 files changed, 447 insertions(+), 463 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 0f5271d476e..3f178eda913 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11937,7 +11937,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12183,7 +12183,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12227,7 +12227,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12387,7 +12387,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 49ed797e8ef..2aee5bb3fe9 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4059,7 +4059,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4134,13 +4134,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4192,7 +4192,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..a7578d8aea6 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -70,7 +70,7 @@ step merge1:
step c1: COMMIT;
step explain_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO target t
USING (SELECT 1 as key, 'merge2a' as val) s
ON s.key = t.key
@@ -175,7 +175,7 @@ step merge1:
step explain_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO target t
USING (SELECT 1 as key, 'merge2a' as val) s
ON s.key = t.key
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..f69359f69ec 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -116,7 +116,7 @@ step "merge2a"
step "explain_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO target t
USING (SELECT 1 as key, 'merge2a' as val) s
ON s.key = t.key
@@ -165,7 +165,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 2d247f73c14..7894b58da8f 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -295,70 +295,75 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
- explain_filter
--------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl", +
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Actual Startup Time": N.N, +
- "Actual Total Time": N.N, +
- "Actual Rows": N.N, +
- "Actual Loops": N, +
- "Disabled": false, +
- "Prefetch": { +
- "Average Distance": N.N, +
- "Max Distance": N, +
- "Capacity": N +
- }, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N, +
- "Shared I/O Read Time": N.N, +
- "Shared I/O Write Time": N.N,+
- "Local I/O Read Time": N.N, +
- "Local I/O Write Time": N.N, +
- "Temp I/O Read Time": N.N, +
- "Temp I/O Write Time": N.N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N, +
- "Shared I/O Read Time": N.N, +
- "Shared I/O Write Time": N.N,+
- "Local I/O Read Time": N.N, +
- "Local I/O Write Time": N.N, +
- "Temp I/O Read Time": N.N, +
- "Temp I/O Write Time": N.N +
- }, +
- "Planning Time": N.N, +
- "Triggers": [ +
- ], +
- "Execution Time": N.N +
- } +
+ explain_filter
+----------------------------------------
+ [ +
+ { +
+ "Plan": { +
+ "Node Type": "Seq Scan", +
+ "Parallel Aware": false, +
+ "Async Capable": false, +
+ "Relation Name": "int8_tbl", +
+ "Alias": "i8", +
+ "Startup Cost": N.N, +
+ "Total Cost": N.N, +
+ "Plan Rows": N, +
+ "Plan Width": N, +
+ "Actual Startup Time": N.N, +
+ "Actual Total Time": N.N, +
+ "Actual Rows": N.N, +
+ "Actual Loops": N, +
+ "Disabled": false, +
+ "Prefetch": { +
+ "Average Distance": N.N, +
+ "Max Distance": N, +
+ "Capacity": N +
+ }, +
+ "I/O": { +
+ "Stalls": N, +
+ "Average IO Size": N.N, +
+ "Average IOs In Progress": N.N+
+ }, +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N, +
+ "Shared I/O Read Time": N.N, +
+ "Shared I/O Write Time": N.N, +
+ "Local I/O Read Time": N.N, +
+ "Local I/O Write Time": N.N, +
+ "Temp I/O Read Time": N.N, +
+ "Temp I/O Write Time": N.N +
+ }, +
+ "Planning": { +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N, +
+ "Shared I/O Read Time": N.N, +
+ "Shared I/O Write Time": N.N, +
+ "Local I/O Read Time": N.N, +
+ "Local I/O Write Time": N.N, +
+ "Temp I/O Read Time": N.N, +
+ "Temp I/O Write Time": N.N +
+ }, +
+ "Planning Time": N.N, +
+ "Triggers": [ +
+ ], +
+ "Execution Time": N.N +
+ } +
]
(1 row)
@@ -435,60 +440,65 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
(1 row)
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Actual Startup Time": N.N, +
- "Actual Total Time": N.N, +
- "Actual Rows": N.N, +
- "Actual Loops": N, +
- "Disabled": false, +
- "Prefetch": { +
- "Average Distance": N.N, +
- "Max Distance": N, +
- "Capacity": N +
- }, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N, +
- "Memory Used": N, +
- "Memory Allocated": N +
- }, +
- "Planning Time": N.N, +
- "Triggers": [ +
- ], +
- "Execution Time": N.N +
- } +
+ explain_filter
+----------------------------------------
+ [ +
+ { +
+ "Plan": { +
+ "Node Type": "Seq Scan", +
+ "Parallel Aware": false, +
+ "Async Capable": false, +
+ "Relation Name": "int8_tbl", +
+ "Alias": "i8", +
+ "Startup Cost": N.N, +
+ "Total Cost": N.N, +
+ "Plan Rows": N, +
+ "Plan Width": N, +
+ "Actual Startup Time": N.N, +
+ "Actual Total Time": N.N, +
+ "Actual Rows": N.N, +
+ "Actual Loops": N, +
+ "Disabled": false, +
+ "Prefetch": { +
+ "Average Distance": N.N, +
+ "Max Distance": N, +
+ "Capacity": N +
+ }, +
+ "I/O": { +
+ "Stalls": N, +
+ "Average IO Size": N.N, +
+ "Average IOs In Progress": N.N+
+ }, +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N +
+ }, +
+ "Planning": { +
+ "Shared Hit Blocks": N, +
+ "Shared Read Blocks": N, +
+ "Shared Dirtied Blocks": N, +
+ "Shared Written Blocks": N, +
+ "Local Hit Blocks": N, +
+ "Local Read Blocks": N, +
+ "Local Dirtied Blocks": N, +
+ "Local Written Blocks": N, +
+ "Temp Read Blocks": N, +
+ "Temp Written Blocks": N, +
+ "Memory Used": N, +
+ "Memory Allocated": N +
+ }, +
+ "Planning Time": N.N, +
+ "Triggers": [ +
+ ], +
+ "Execution Time": N.N +
+ } +
]
(1 row)
@@ -552,173 +562,178 @@ select jsonb_pretty(
#- '{0,Plan,Plans,0,Sort Method}'
#- '{0,Plan,Plans,0,Sort Space Type}'
);
- jsonb_pretty
--------------------------------------------------------------
- [ +
- { +
- "Plan": { +
- "Plans": [ +
- { +
- "Plans": [ +
- { +
- "Alias": "tenk1", +
- "Output": [ +
- "unique1", +
- "unique2", +
- "two", +
- "four", +
- "ten", +
- "twenty", +
- "hundred", +
- "thousand", +
- "twothousand", +
- "fivethous", +
- "tenthous", +
- "odd", +
- "even", +
- "stringu1", +
- "stringu2", +
- "string4" +
- ], +
- "Schema": "public", +
- "Disabled": false, +
- "Prefetch": { +
- "Capacity": 0, +
- "Max Distance": 0, +
- "Average Distance": 0.0 +
- }, +
- "Node Type": "Seq Scan", +
- "Plan Rows": 0, +
- "Plan Width": 0, +
- "Total Cost": 0.0, +
- "Actual Rows": 0.0, +
- "Actual Loops": 0, +
- "Startup Cost": 0.0, +
- "Async Capable": false, +
- "Relation Name": "tenk1", +
- "Parallel Aware": true, +
- "Local Hit Blocks": 0, +
- "Temp Read Blocks": 0, +
- "Actual Total Time": 0.0, +
- "Local Read Blocks": 0, +
- "Shared Hit Blocks": 0, +
- "Shared Read Blocks": 0, +
- "Actual Startup Time": 0.0, +
- "Parent Relationship": "Outer",+
- "Temp Written Blocks": 0, +
- "Local Dirtied Blocks": 0, +
- "Local Written Blocks": 0, +
- "Shared Dirtied Blocks": 0, +
- "Shared Written Blocks": 0 +
- } +
- ], +
- "Output": [ +
- "unique1", +
- "unique2", +
- "two", +
- "four", +
- "ten", +
- "twenty", +
- "hundred", +
- "thousand", +
- "twothousand", +
- "fivethous", +
- "tenthous", +
- "odd", +
- "even", +
- "stringu1", +
- "stringu2", +
- "string4" +
- ], +
- "Disabled": false, +
- "Sort Key": [ +
- "tenk1.tenthous" +
- ], +
- "Node Type": "Sort", +
- "Plan Rows": 0, +
- "Plan Width": 0, +
- "Total Cost": 0.0, +
- "Actual Rows": 0.0, +
- "Actual Loops": 0, +
- "Startup Cost": 0.0, +
- "Async Capable": false, +
- "Parallel Aware": false, +
- "Sort Space Used": 0, +
- "Local Hit Blocks": 0, +
- "Temp Read Blocks": 0, +
- "Actual Total Time": 0.0, +
- "Local Read Blocks": 0, +
- "Shared Hit Blocks": 0, +
- "Shared Read Blocks": 0, +
- "Actual Startup Time": 0.0, +
- "Parent Relationship": "Outer", +
- "Temp Written Blocks": 0, +
- "Local Dirtied Blocks": 0, +
- "Local Written Blocks": 0, +
- "Shared Dirtied Blocks": 0, +
- "Shared Written Blocks": 0 +
- } +
- ], +
- "Output": [ +
- "unique1", +
- "unique2", +
- "two", +
- "four", +
- "ten", +
- "twenty", +
- "hundred", +
- "thousand", +
- "twothousand", +
- "fivethous", +
- "tenthous", +
- "odd", +
- "even", +
- "stringu1", +
- "stringu2", +
- "string4" +
- ], +
- "Disabled": false, +
- "Node Type": "Gather Merge", +
- "Plan Rows": 0, +
- "Plan Width": 0, +
- "Total Cost": 0.0, +
- "Actual Rows": 0.0, +
- "Actual Loops": 0, +
- "Startup Cost": 0.0, +
- "Async Capable": false, +
- "Parallel Aware": false, +
- "Workers Planned": 0, +
- "Local Hit Blocks": 0, +
- "Temp Read Blocks": 0, +
- "Workers Launched": 0, +
- "Actual Total Time": 0.0, +
- "Local Read Blocks": 0, +
- "Shared Hit Blocks": 0, +
- "Shared Read Blocks": 0, +
- "Actual Startup Time": 0.0, +
- "Temp Written Blocks": 0, +
- "Local Dirtied Blocks": 0, +
- "Local Written Blocks": 0, +
- "Shared Dirtied Blocks": 0, +
- "Shared Written Blocks": 0 +
- }, +
- "Planning": { +
- "Local Hit Blocks": 0, +
- "Temp Read Blocks": 0, +
- "Local Read Blocks": 0, +
- "Shared Hit Blocks": 0, +
- "Shared Read Blocks": 0, +
- "Temp Written Blocks": 0, +
- "Local Dirtied Blocks": 0, +
- "Local Written Blocks": 0, +
- "Shared Dirtied Blocks": 0, +
- "Shared Written Blocks": 0 +
- }, +
- "Triggers": [ +
- ], +
- "Planning Time": 0.0, +
- "Execution Time": 0.0 +
- } +
+ jsonb_pretty
+----------------------------------------------------------------
+ [ +
+ { +
+ "Plan": { +
+ "Plans": [ +
+ { +
+ "Plans": [ +
+ { +
+ "I/O": { +
+ "Stalls": 0, +
+ "Average IO Size": 0.0, +
+ "Average IOs In Progress": 0.0+
+ }, +
+ "Alias": "tenk1", +
+ "Output": [ +
+ "unique1", +
+ "unique2", +
+ "two", +
+ "four", +
+ "ten", +
+ "twenty", +
+ "hundred", +
+ "thousand", +
+ "twothousand", +
+ "fivethous", +
+ "tenthous", +
+ "odd", +
+ "even", +
+ "stringu1", +
+ "stringu2", +
+ "string4" +
+ ], +
+ "Schema": "public", +
+ "Disabled": false, +
+ "Prefetch": { +
+ "Capacity": 0, +
+ "Max Distance": 0, +
+ "Average Distance": 0.0 +
+ }, +
+ "Node Type": "Seq Scan", +
+ "Plan Rows": 0, +
+ "Plan Width": 0, +
+ "Total Cost": 0.0, +
+ "Actual Rows": 0.0, +
+ "Actual Loops": 0, +
+ "Startup Cost": 0.0, +
+ "Async Capable": false, +
+ "Relation Name": "tenk1", +
+ "Parallel Aware": true, +
+ "Local Hit Blocks": 0, +
+ "Temp Read Blocks": 0, +
+ "Actual Total Time": 0.0, +
+ "Local Read Blocks": 0, +
+ "Shared Hit Blocks": 0, +
+ "Shared Read Blocks": 0, +
+ "Actual Startup Time": 0.0, +
+ "Parent Relationship": "Outer", +
+ "Temp Written Blocks": 0, +
+ "Local Dirtied Blocks": 0, +
+ "Local Written Blocks": 0, +
+ "Shared Dirtied Blocks": 0, +
+ "Shared Written Blocks": 0 +
+ } +
+ ], +
+ "Output": [ +
+ "unique1", +
+ "unique2", +
+ "two", +
+ "four", +
+ "ten", +
+ "twenty", +
+ "hundred", +
+ "thousand", +
+ "twothousand", +
+ "fivethous", +
+ "tenthous", +
+ "odd", +
+ "even", +
+ "stringu1", +
+ "stringu2", +
+ "string4" +
+ ], +
+ "Disabled": false, +
+ "Sort Key": [ +
+ "tenk1.tenthous" +
+ ], +
+ "Node Type": "Sort", +
+ "Plan Rows": 0, +
+ "Plan Width": 0, +
+ "Total Cost": 0.0, +
+ "Actual Rows": 0.0, +
+ "Actual Loops": 0, +
+ "Startup Cost": 0.0, +
+ "Async Capable": false, +
+ "Parallel Aware": false, +
+ "Sort Space Used": 0, +
+ "Local Hit Blocks": 0, +
+ "Temp Read Blocks": 0, +
+ "Actual Total Time": 0.0, +
+ "Local Read Blocks": 0, +
+ "Shared Hit Blocks": 0, +
+ "Shared Read Blocks": 0, +
+ "Actual Startup Time": 0.0, +
+ "Parent Relationship": "Outer", +
+ "Temp Written Blocks": 0, +
+ "Local Dirtied Blocks": 0, +
+ "Local Written Blocks": 0, +
+ "Shared Dirtied Blocks": 0, +
+ "Shared Written Blocks": 0 +
+ } +
+ ], +
+ "Output": [ +
+ "unique1", +
+ "unique2", +
+ "two", +
+ "four", +
+ "ten", +
+ "twenty", +
+ "hundred", +
+ "thousand", +
+ "twothousand", +
+ "fivethous", +
+ "tenthous", +
+ "odd", +
+ "even", +
+ "stringu1", +
+ "stringu2", +
+ "string4" +
+ ], +
+ "Disabled": false, +
+ "Node Type": "Gather Merge", +
+ "Plan Rows": 0, +
+ "Plan Width": 0, +
+ "Total Cost": 0.0, +
+ "Actual Rows": 0.0, +
+ "Actual Loops": 0, +
+ "Startup Cost": 0.0, +
+ "Async Capable": false, +
+ "Parallel Aware": false, +
+ "Workers Planned": 0, +
+ "Local Hit Blocks": 0, +
+ "Temp Read Blocks": 0, +
+ "Workers Launched": 0, +
+ "Actual Total Time": 0.0, +
+ "Local Read Blocks": 0, +
+ "Shared Hit Blocks": 0, +
+ "Shared Read Blocks": 0, +
+ "Actual Startup Time": 0.0, +
+ "Temp Written Blocks": 0, +
+ "Local Dirtied Blocks": 0, +
+ "Local Written Blocks": 0, +
+ "Shared Dirtied Blocks": 0, +
+ "Shared Written Blocks": 0 +
+ }, +
+ "Planning": { +
+ "Local Hit Blocks": 0, +
+ "Temp Read Blocks": 0, +
+ "Local Read Blocks": 0, +
+ "Shared Hit Blocks": 0, +
+ "Shared Read Blocks": 0, +
+ "Temp Written Blocks": 0, +
+ "Local Dirtied Blocks": 0, +
+ "Local Written Blocks": 0, +
+ "Shared Dirtied Blocks": 0, +
+ "Shared Written Blocks": 0 +
+ }, +
+ "Triggers": [ +
+ ], +
+ "Planning Time": 0.0, +
+ "Execution Time": 0.0 +
+ } +
]
(1 row)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1332cd0e646..23e8627e5a5 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2169,7 +2169,7 @@ create table ab_a3_b3 partition of ab_a3 for values in (3);
set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2, 3);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2182,7 +2182,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1
Filter: ((a >= $1) AND (a <= $2) AND (b <= $3))
(8 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (1, 2, 3);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2205,7 +2205,7 @@ deallocate ab_q1;
-- Runtime pruning after optimizer pruning
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2216,7 +2216,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1
Filter: ((a >= $1) AND (a <= $2) AND (b < 3))
(6 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 4);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2235,7 +2235,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q2 (2, 2);
QUERY PLAN
----------------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2253,7 +2253,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q3 (2, 2);
QUERY PLAN
----------------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2315,57 +2315,48 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
Subplans Removed: 3
-> Seq Scan on list_part1 list_part_1 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(1))
- Prefetch: avg=1.000 max=1 capacity=94
-(5 rows)
+(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
-> Seq Scan on list_part1 list_part_1 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part2 list_part_2 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part3 list_part_3 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part4 list_part_4 (actual rows=1.00 loops=1)
Filter: (a = list_part_fn(a))
- Prefetch: avg=1.000 max=1 capacity=94
-(13 rows)
+(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
-> Seq Scan on list_part1 list_part_1 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part2 list_part_2 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part3 list_part_3 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on list_part4 list_part_4 (actual rows=0.00 loops=1)
Filter: (a = (list_part_fn(1) + a))
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-(17 rows)
+(13 rows)
rollback;
drop table list_part;
@@ -2385,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2533,7 +2524,6 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Nested Loop (actual rows=N loops=N)
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{0,0,1}'::integer[]))
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = a.a)
@@ -2553,7 +2543,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = a.a)
-(28 rows)
+(27 rows)
-- Ensure the same partitions are pruned when we make the nested loop
-- parameter an Expr rather than a plain Param.
@@ -2568,7 +2558,6 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Nested Loop (actual rows=N loops=N)
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{0,0,1}'::integer[]))
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = (a.a + 0))
@@ -2588,7 +2577,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = (a.a + 0))
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = (a.a + 0))
-(28 rows)
+(27 rows)
insert into lprt_a values(3),(3);
select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on ab.a = a.a where a.a in(1, 0, 3)');
@@ -2602,7 +2591,6 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Nested Loop (actual rows=N loops=N)
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{1,0,3}'::integer[]))
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = a.a)
@@ -2622,7 +2610,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (actual rows=N loops=N)
Index Cond: (a = a.a)
-(28 rows)
+(27 rows)
select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on ab.a = a.a where a.a in(1, 0, 0)');
explain_parallel_append
@@ -2636,7 +2624,6 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{1,0,0}'::integer[]))
Rows Removed by Filter: N
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (actual rows=N loops=N)
Index Cond: (a = a.a)
@@ -2656,7 +2643,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = a.a)
-(29 rows)
+(28 rows)
delete from lprt_a where a = 1;
select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on ab.a = a.a where a.a in(1, 0, 0)');
@@ -2671,7 +2658,6 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
-> Parallel Seq Scan on lprt_a a (actual rows=N loops=N)
Filter: (a = ANY ('{1,0,0}'::integer[]))
Rows Removed by Filter: N
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=N loops=N)
-> Index Scan using ab_a1_b1_a_idx on ab_a1_b1 ab_1 (never executed)
Index Cond: (a = a.a)
@@ -2691,7 +2677,7 @@ select explain_parallel_append('select avg(ab.a) from ab inner join lprt_a a on
Index Cond: (a = a.a)
-> Index Scan using ab_a3_b3_a_idx on ab_a3_b3 ab_9 (never executed)
Index Cond: (a = a.a)
-(29 rows)
+(28 rows)
reset enable_hashjoin;
reset enable_mergejoin;
@@ -2701,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2709,11 +2695,9 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
InitPlan expr_1
-> Aggregate (actual rows=1.00 loops=1)
-> Seq Scan on lprt_a (actual rows=102.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
InitPlan expr_2
-> Aggregate (actual rows=1.00 loops=1)
-> Seq Scan on lprt_a lprt_a_1 (actual rows=102.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Bitmap Heap Scan on ab_a1_b1 ab_1 (never executed)
Recheck Cond: (a = (InitPlan expr_1).col1)
Filter: (b = (InitPlan expr_2).col1)
@@ -2768,10 +2752,10 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
-> Bitmap Index Scan on ab_a3_b3_a_idx (never executed)
Index Cond: (a = (InitPlan expr_1).col1)
Index Searches: 0
-(63 rows)
+(61 rows)
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
QUERY PLAN
----------------------------------------------------------------------------------
@@ -2818,7 +2802,7 @@ select * from (select * from ab where a = 1 union all select * from ab) ab where
(40 rows)
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
QUERY PLAN
----------------------------------------------------------------------------------
@@ -2880,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2896,14 +2880,13 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6
-> Seq Scan on xy_1 (actual rows=0.00 loops=1)
Filter: ((x = $1) AND (y = (InitPlan expr_1).col1))
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on ab_a1_b1 ab_4 (never executed)
Filter: ((a = $1) AND (b = (InitPlan expr_1).col1))
-> Seq Scan on ab_a1_b2 ab_5 (never executed)
Filter: ((a = $1) AND (b = (InitPlan expr_1).col1))
-> Seq Scan on ab_a1_b3 ab_6 (never executed)
Filter: ((a = $1) AND (b = (InitPlan expr_1).col1))
-(20 rows)
+(19 rows)
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -3036,13 +3019,12 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
Nested Loop (actual rows=6.00 loops=1)
-> Seq Scan on tbl1 (actual rows=2.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=3.00 loops=2)
-> Index Scan using tprt1_idx on tprt_1 (actual rows=2.00 loops=2)
Index Cond: (col1 < tbl1.col1)
@@ -3062,15 +3044,14 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 < tbl1.col1)
Index Searches: 0
-(22 rows)
+(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
Nested Loop (actual rows=2.00 loops=1)
-> Seq Scan on tbl1 (actual rows=2.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=1.00 loops=2)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 = tbl1.col1)
@@ -3090,7 +3071,7 @@ select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 = tbl1.col1)
Index Searches: 0
-(22 rows)
+(21 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 > tprt.col1
@@ -3116,13 +3097,12 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
Nested Loop (actual rows=23.00 loops=1)
-> Seq Scan on tbl1 (actual rows=5.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=4.60 loops=5)
-> Index Scan using tprt1_idx on tprt_1 (actual rows=2.00 loops=5)
Index Cond: (col1 < tbl1.col1)
@@ -3142,15 +3122,14 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 < tbl1.col1)
Index Searches: 0
-(22 rows)
+(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
Nested Loop (actual rows=3.00 loops=1)
-> Seq Scan on tbl1 (actual rows=5.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=0.60 loops=5)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 = tbl1.col1)
@@ -3170,7 +3149,7 @@ select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 = tbl1.col1)
Index Searches: 0
-(22 rows)
+(21 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 > tprt.col1
@@ -3215,13 +3194,12 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
Nested Loop (actual rows=1.00 loops=1)
-> Seq Scan on tbl1 (actual rows=1.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=1.00 loops=1)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 > tbl1.col1)
@@ -3241,7 +3219,7 @@ select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (actual rows=1.00 loops=1)
Index Cond: (col1 > tbl1.col1)
Index Searches: 1
-(22 rows)
+(21 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 < tprt.col1
@@ -3254,13 +3232,12 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
Nested Loop (actual rows=0.00 loops=1)
-> Seq Scan on tbl1 (actual rows=1.00 loops=1)
- Prefetch: avg=1.000 max=1 capacity=94
-> Append (actual rows=0.00 loops=1)
-> Index Scan using tprt1_idx on tprt_1 (never executed)
Index Cond: (col1 = tbl1.col1)
@@ -3280,7 +3257,7 @@ select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
-> Index Scan using tprt6_idx on tprt_6 (never executed)
Index Cond: (col1 = tbl1.col1)
Index Searches: 0
-(22 rows)
+(21 rows)
select tbl1.col1, tprt.col1 from tbl1
inner join tprt on tbl1.col1 = tprt.col1
@@ -3301,7 +3278,7 @@ alter table part_cab attach partition part_abc_p1 for values in(3);
prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute part_abc_q1 (1, 2, 3);
QUERY PLAN
-------------------------------------------------------------
Seq Scan on part_abc_p1 part_abc (actual rows=0.00 loops=1)
@@ -3326,7 +3303,7 @@ select * from listp where b = 1;
-- partitions before finally detecting the correct set of 2nd level partitions
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,1);
QUERY PLAN
----------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3335,7 +3312,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
Filter: (b = ANY (ARRAY[$1, $2]))
(4 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (2,2);
QUERY PLAN
----------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3345,7 +3322,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
(4 rows)
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (0,0);
QUERY PLAN
-----------------------------------
Append (actual rows=0.00 loops=1)
@@ -3356,7 +3333,7 @@ deallocate q1;
-- Test more complex cases where a not-equal condition further eliminates partitions.
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,0);
QUERY PLAN
-------------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3366,7 +3343,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
(4 rows)
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,1);
QUERY PLAN
-----------------------------------
Append (actual rows=0.00 loops=1)
@@ -3374,7 +3351,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
(2 rows)
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select null::int);
QUERY PLAN
------------------------------------------------------
@@ -3399,7 +3376,7 @@ create table stable_qual_pruning2 partition of stable_qual_pruning
create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < localtimestamp;
QUERY PLAN
-----------------------------------------------------------------------------------------
@@ -3412,7 +3389,7 @@ select * from stable_qual_pruning where a < localtimestamp;
(6 rows)
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
QUERY PLAN
-----------------------------------------------------------------------------------------
@@ -3423,7 +3400,7 @@ select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
(4 rows)
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
QUERY PLAN
@@ -3433,7 +3410,7 @@ select * from stable_qual_pruning
One-Time Filter: false
(3 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
QUERY PLAN
@@ -3442,7 +3419,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000","Fri Jan 01 00:00:00 2010"}'::timestamp without time zone[]))
(2 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
QUERY PLAN
@@ -3453,7 +3430,7 @@ select * from stable_qual_pruning
Filter: (a = ANY (ARRAY['Tue Feb 01 00:00:00 2000'::timestamp without time zone, LOCALTIMESTAMP]))
(4 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
QUERY PLAN
@@ -3462,7 +3439,7 @@ select * from stable_qual_pruning
Subplans Removed: 3
(2 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
QUERY PLAN
@@ -3473,7 +3450,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000 PST","Fri Jan 01 00:00:00 2010 PST"}'::timestamp with time zone[]))
(4 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
QUERY PLAN
@@ -3501,21 +3478,18 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
Append (actual rows=3.00 loops=1)
-> Seq Scan on mc3p0 mc3p_1 (actual rows=1.00 loops=1)
Filter: ((a < 3) AND (abs(b) = 1))
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on mc3p1 mc3p_2 (actual rows=1.00 loops=1)
Filter: ((a < 3) AND (abs(b) = 1))
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on mc3p2 mc3p_3 (actual rows=1.00 loops=1)
Filter: ((a < 3) AND (abs(b) = 1))
- Prefetch: avg=1.000 max=1 capacity=94
-(10 rows)
+(7 rows)
--
-- Check that pruning with composite range partitioning works correctly when
@@ -3524,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3534,13 +3508,12 @@ execute ps1(1);
-> Result (actual rows=1.00 loops=1)
-> Seq Scan on mc3p1 mc3p_1 (actual rows=1.00 loops=1)
Filter: ((a = $1) AND (abs(b) < (InitPlan expr_1).col1))
- Prefetch: avg=1.000 max=1 capacity=94
-(7 rows)
+(6 rows)
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3550,11 +3523,9 @@ execute ps2(1);
-> Result (actual rows=1.00 loops=1)
-> Seq Scan on mc3p0 mc3p_1 (actual rows=1.00 loops=1)
Filter: ((a <= $1) AND (abs(b) < (InitPlan expr_1).col1))
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on mc3p1 mc3p_2 (actual rows=1.00 loops=1)
Filter: ((a <= $1) AND (abs(b) < (InitPlan expr_1).col1))
- Prefetch: avg=1.000 max=1 capacity=94
-(10 rows)
+(8 rows)
deallocate ps2;
drop table mc3p;
@@ -3564,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3573,14 +3544,13 @@ select * from boolp where a = (select value from boolvalues where value);
-> Seq Scan on boolvalues (actual rows=1.00 loops=1)
Filter: value
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on boolp_f boolp_1 (never executed)
Filter: (a = (InitPlan expr_1).col1)
-> Seq Scan on boolp_t boolp_2 (actual rows=0.00 loops=1)
Filter: (a = (InitPlan expr_1).col1)
-(10 rows)
+(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
@@ -3589,12 +3559,11 @@ select * from boolp where a = (select value from boolvalues where not value);
-> Seq Scan on boolvalues (actual rows=1.00 loops=1)
Filter: (NOT value)
Rows Removed by Filter: 1
- Prefetch: avg=1.000 max=1 capacity=94
-> Seq Scan on boolp_f boolp_1 (actual rows=0.00 loops=1)
Filter: (a = (InitPlan expr_1).col1)
-> Seq Scan on boolp_t boolp_2 (never executed)
Filter: (a = (InitPlan expr_1).col1)
-(10 rows)
+(9 rows)
drop table boolp;
--
@@ -3610,7 +3579,7 @@ insert into ma_test select x,x from generate_series(0,29) t(x);
create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(15);
QUERY PLAN
--------------------------------------------------------------------------------------------
Merge Append (actual rows=2.00 loops=1)
@@ -3633,7 +3602,7 @@ execute mt_q1(15);
25
(2 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(25);
QUERY PLAN
--------------------------------------------------------------------------------------------
Merge Append (actual rows=1.00 loops=1)
@@ -3652,7 +3621,7 @@ execute mt_q1(25);
(1 row)
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(35);
QUERY PLAN
-----------------------------------------
Merge Append (actual rows=0.00 loops=1)
@@ -3680,7 +3649,7 @@ explain (analyze, verbose, costs off, summary off, timing off, buffers off) exec
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
QUERY PLAN
--------------------------------------------------------------------------------------------------
Merge Append (actual rows=20.00 loops=1)
@@ -4139,7 +4108,7 @@ create table listp (a int, b int) partition by list (a);
create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select 2) and b <> 10;
QUERY PLAN
--------------------------------------------------------
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..0934cca73f6 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -480,8 +480,8 @@ set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (1, 2, 3);
deallocate ab_q1;
@@ -489,21 +489,21 @@ deallocate ab_q1;
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 4);
-- Ensure a mix of PARAM_EXTERN and PARAM_EXEC Params work together at
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q3 (2, 2);
--
-- Test runtime pruning with hash partitioned tables
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,15 +669,15 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
-- Another UNION ALL test, but containing a mix of exec init and exec run-time pruning.
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -818,7 +818,7 @@ prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute part_abc_q1 (1, 2, 3);
deallocate part_abc_q1;
@@ -838,12 +838,12 @@ select * from listp where b = 1;
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,1);
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (2,2);
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (0,0);
deallocate q1;
@@ -851,13 +851,13 @@ deallocate q1;
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,0);
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,1);
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select null::int);
drop table listp;
@@ -874,30 +874,30 @@ create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < localtimestamp;
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
@@ -969,12 +969,12 @@ create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(15);
execute mt_q1(15);
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(25);
execute mt_q1(25);
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(35);
execute mt_q1(35);
deallocate mt_q1;
@@ -987,7 +987,7 @@ explain (analyze, verbose, costs off, summary off, timing off, buffers off) exec
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
reset enable_seqscan;
reset enable_sort;
@@ -1167,7 +1167,7 @@ create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select 2) and b <> 10;
--
--
2.47.1
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
@ 2026-03-19 20:57 ` Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-19 20:57 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Lukas,
Thanks for the review. I've been working on cleaning up the patch,
reordering the changes a bit. I ran into most of the issues you
mentioned, and the attached v4 should address most of them.
I got rid of the "old" approach entirely, and just do everything through
the scan descriptor. I reorganized the patch series to make more sense -
it's split into three pieces (0001 is a prerequisite patch, not really a
part of the changes):
* 0002 - Basic changes, adding the info to a single scan node that
already uses shared instrumentation (=BitmapHeapScan). It adds the
counting to read_stream, and printing to explain. It also adjusts all
the failing tests checking query plans (usually by disabling I/O).
* 0003 - Adds support for SeqScan (has to setup the instrumentation for
parallel workers, ...).
* 0004 - Adds support for TidRangeScan (has to setup the instrumentation
for parallel workers, ...).
I think it'd make sense to commit these parts separately.
The 0002 part could be made smaller by introducing a new read_stream
helper, so that we don't need to add a parameter to all the places
calling read_stream_begin_relation(). But that's a detail.
More comments inline.
On 3/19/26 19:11, Lukas Fittl wrote:
> On Wed, Mar 18, 2026 at 3:41 PM Tomas Vondra <tomas@vondra.me> wrote:
>> The 0003 also changes the EXPLAIN to enable IO by default, just like we
>> do for BUFFERS. It seems like a reasonable precedent to me.
>
> One side effect of that is that the tests now fail for me locally,
> because the specific values are system-dependent. Attached a patch
> (nocfbot-0002) that fixed that for me.
>
Right. I haven't addressed that in v3, but with v4 the tests should be
passing (after each patch).
> There is one detail maybe calling out specifically on JSON output:
> Currently Postgres always emits all fields in JSON output, even if
> they are zero. The code that you have in v3 skips the "I/O" group when
> the value is zero, which doesn't work well with how current regression
> tests are written. I'm definitely not a fan of the unnecessary
> verbosity of JSON EXPLAIN output, but I'd suggest we don't break with
> the tradition here, and instead always output the "I/O" group in
> non-text formats. Also attached a patch for that (nocfbot-0001).
>
Yeah, I noticed that too, but I wasn't sure what to do about it. The I/O
stats are conditional on (io_count > 0) because it calculates average
I/O size etc. If there are no I/Os, we could probably print 0.0.
Another thing I noticed in the non-text formats is that this is the only
case with explicit groups, which adds nesting. It looks weird, so I plan
to remove that, and will add some "prefix" to the items.
> Overall I think the abstraction here seems reasonable if we're
> primarily focused on getting the per-node instrumentation taken care
> of.
>
OK, good. I think passing the stats through a scan descriptor works OK.
I still have a strange feeling about it - we have the TAM interface and
yet we choose to pass data in other ways. It's better than global
variables, of course. But it's not the only piece of data returned
through the scan descriptor, so at least there's a precedent.
> That said, two thoughts on an example EXPLAIN output I just ran:
>
> 1) I do wonder if its a bit confusing that we propagate I/O timings up
> the EXPLAIN tree, but not the "I/O" information - I realize fixing
> that would be a bit involved though, e.g. we'd have to invent
> accumulation logic in explain.c. It'd also maybe make people thing
> this covers things like temporary file reads/etc.
>
I'm not sure it makes sense to combine this information from multiple
nodes. If you have multiple read streams with very different average I/O
size and/or prefetch distances, what does the "global average" mean?
I also don't think it's very useful - the information about distance
and/or I/O size is most useful when analyzing individual nodes. What
would it tell me for multiple nodes combined?
So I don't think it's something we need, and we can add it later.
> 2) The ordering of "I/O Timings" in relation to "I/O" feels off to me
> (since they're not next to each other) - maybe we should re-order I/O
> Timings to come before Buffers in show_buffer_usage to address that?
>
Yeah, maybe. I don't think we have any explicit order, but why not.
Thanks!
--
Tomas Vondra
Attachments:
[text/x-patch] v4-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch (2.8K, ../../2551c884-af06-4663-83d2-cd5688c3ae12@vondra.me/2-v4-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch)
download | inline diff:
From 2c3fab386d4613a9b4b735fbc887b063edba61ec Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH v4 1/4] bufmgr: Return whether WaitReadBuffers() needed to
wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 00bc609529a..c8ec7fb4f53 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1738,12 +1738,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1805,6 +1813,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1835,6 +1844,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1851,6 +1866,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4017896f951..a1ef04354dd 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -249,7 +249,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.53.0
[text/x-patch] v4-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch (69.7K, ../../2551c884-af06-4663-83d2-cd5688c3ae12@vondra.me/3-v4-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch)
download | inline diff:
From 5d9fbb19d27d5866646ceea162afbb9686af73e8 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Thu, 12 Mar 2026 02:04:07 +0100
Subject: [PATCH v4 2/4] explain: show prefetch stats in EXPLAIN (ANALYZE)
This adds details about AIO / prefetch for executor nodes using a
ReadStream. Right now this applies only to BitmapHeapScan, because
that's the only scan node using a ReadStream and collecting
instrumentation from workers.
The stats are collected by the ReadStream unconditionally, i.e. without
EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
integer counters), it's not worth gating this by a flag.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) so that the executor can access it and use it for explain.
The collected stats are designed for ReadStream, but is meant to be
reasonably generic in case a TAM manages I/Os in different ways.
In EXPLAIN, the statistic is triggered by a new option "IO", enabled by
default. This mimics what we do for BUFFERS, which seems like the
closest EXPLAIN option.
---
contrib/amcheck/verify_heapam.c | 3 +-
contrib/bloom/blscan.c | 3 +-
contrib/bloom/blvacuum.c | 6 +-
contrib/pg_prewarm/autoprewarm.c | 3 +-
contrib/pg_prewarm/pg_prewarm.c | 3 +-
contrib/pg_visibility/pg_visibility.c | 6 +-
contrib/pgstattuple/pgstatapprox.c | 3 +-
contrib/pgstattuple/pgstatindex.c | 6 +-
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/brin/brin.c | 3 +-
src/backend/access/gin/ginvacuum.c | 3 +-
src/backend/access/gist/gistvacuum.c | 3 +-
src/backend/access/hash/hash.c | 3 +-
src/backend/access/heap/heapam.c | 9 +-
src/backend/access/heap/vacuumlazy.c | 6 +-
src/backend/access/nbtree/nbtree.c | 3 +-
src/backend/access/spgist/spgvacuum.c | 3 +-
src/backend/commands/analyze.c | 3 +-
src/backend/commands/explain.c | 162 +++++++++++++++++-
src/backend/commands/explain_state.c | 15 ++
src/backend/executor/nodeBitmapHeapscan.c | 3 +
src/backend/storage/aio/read_stream.c | 91 +++++++++-
src/include/access/relscan.h | 6 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 42 +++++
src/include/storage/read_stream.h | 4 +-
.../modules/index/expected/killtuples.out | 44 ++---
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/regress/expected/explain.out | 20 +--
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 2 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 20 +--
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
42 files changed, 436 insertions(+), 91 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index ada782f98f5..0ec4dc849c3 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -476,7 +476,8 @@ verify_heapam(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
stream_cb,
stream_data,
- 0);
+ 0,
+ NULL);
while ((ctx.buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
{
diff --git a/contrib/bloom/blscan.c b/contrib/bloom/blscan.c
index 1a0e42021ec..9bdc4a95c18 100644
--- a/contrib/bloom/blscan.c
+++ b/contrib/bloom/blscan.c
@@ -138,7 +138,8 @@ blgetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
index 6beb1c20ebb..587757c1e18 100644
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -71,7 +71,8 @@ blbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
@@ -224,7 +225,8 @@ blvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index ba0bc8e6d4a..4c5448f177a 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -628,7 +628,8 @@ autoprewarm_database_main(Datum main_arg)
p.forknum,
apw_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/*
* Loop until we've prewarmed all the blocks from this fork. The
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index c2716086693..e4541f46777 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -256,7 +256,8 @@ pg_prewarm(PG_FUNCTION_ARGS)
forkNumber,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (block = first_block; block <= last_block; ++block)
{
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dfab0b64cf5..f09bb68624f 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -519,7 +519,8 @@ collect_visibility_data(Oid relid, bool include_pd)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
}
for (blkno = 0; blkno < nblocks; ++blkno)
@@ -739,7 +740,8 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen)
MAIN_FORKNUM,
collect_corrupt_items_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/* Loop over every block in the relation. */
while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 21e0b50fb4b..a926a1f1168 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -146,7 +146,8 @@ statapprox_heap(Relation rel, output_type *stat)
MAIN_FORKNUM,
statapprox_heap_read_stream_next,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 3a3f2637bd9..0de33558c47 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -296,7 +296,8 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
@@ -685,7 +686,8 @@ pgstathashindex(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..4951b624893 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>TRUE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 2a0f8c8e3b8..eb4e4c5de4a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2192,7 +2192,8 @@ brin_vacuum_scan(Relation idxrel, BufferAccessStrategy strategy)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* Scan the index in physical order, and clean up any possible mess in
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 840543eb664..484360a9d6d 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -799,7 +799,8 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = GIN_ROOT_BLKNO; blkno < npages; blkno++)
{
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 686a0418054..e4f578b2de0 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -223,7 +223,8 @@ gistvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 6df5e7ccbd1..043088dbee8 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -546,7 +546,8 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
hash_bulkdelete_read_stream_cb,
&stream_private,
- 0);
+ 0,
+ NULL);
bucket_loop:
while (cur_bucket <= cur_maxbucket)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e5bd062de77..6acbcf1f342 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1267,6 +1267,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_read_stream = NULL;
+ /* reset the stats counters */
+ memset(&scan->rs_base.rs_iostats, 0, sizeof(IOStats));
+
/*
* Set up a read stream for sequential scans and TID range scans. This
* should be done after initscan() because initscan() allocates the
@@ -1296,7 +1299,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
cb,
scan,
- 0);
+ 0,
+ &scan->rs_base.rs_iostats);
}
else if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
{
@@ -1307,7 +1311,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
bitmapheap_stream_read_next,
scan,
- sizeof(TBMIterateResult));
+ sizeof(TBMIterateResult),
+ &scan->rs_base.rs_iostats);
}
scan->rs_vmbuffer = InvalidBuffer;
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 82c5b28e0ad..bec454de6d8 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1290,7 +1290,8 @@ lazy_scan_heap(LVRelState *vacrel)
MAIN_FORKNUM,
heap_vac_scan_next_block,
vacrel,
- sizeof(bool));
+ sizeof(bool),
+ NULL);
while (true)
{
@@ -2790,7 +2791,8 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
MAIN_FORKNUM,
vacuum_reap_lp_read_stream_next,
iter,
- sizeof(TidStoreIterResult));
+ sizeof(TidStoreIterResult),
+ NULL);
while (true)
{
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index aed74590cf4..e5cd51cce4d 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -1325,7 +1325,8 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index 6b7117b56b2..206a8e9c8b0 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -835,7 +835,8 @@ spgvacuumscan(spgBulkDeleteState *bds)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* The outer loop iterates over all index pages except the metapage, in
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index eeed91be266..a2cf176a485 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -1251,7 +1251,8 @@ acquire_sample_rows(Relation onerel, int elevel,
MAIN_FORKNUM,
block_sampling_read_stream_next,
&bs,
- 0);
+ 0,
+ NULL);
/* Outer loop over blocks to sample */
while (table_scan_analyze_next_block(scan, stream))
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 296ea8a1ed2..d98962f50b3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,8 @@
*/
#include "postgres.h"
+#include "access/genam.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +141,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
+static void show_io_usage(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2016,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2035,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2297,8 +2306,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->walusage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
@@ -2315,6 +2324,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+ if (es->io)
+ show_io_usage(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3985,6 +3997,152 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats->stall_count);
+
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "Prefetch", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainCloseGroup("Prefetch", "Prefetch", true, es);
+
+ if (stats->io_count > 0)
+ {
+ ExplainOpenGroup("I/O", "I/O", true, es);
+
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+
+ ExplainCloseGroup("I/O", "I/O", true, es);
+ }
+ }
+ }
+}
+
+/*
+ * show_scan_io_usage
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started, no prefetch stats */
+ if (!(planstate && planstate->ss_currentScanDesc))
+ return;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_iostats;
+
+ /* Initialize counters with stats from the local process first */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->io);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
+/*
+ * show_io_usage
+ * show info about I/O prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_io_usage(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ IOStats *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->io;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -159,6 +160,11 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ io_set = true;
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -179,12 +185,21 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7cf8d23c742..b785cb7a237 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -328,6 +328,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->io, &node->ss.ss_currentScanDesc->rs_iostats);
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74ac..56b0b8e7fb5 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -107,6 +108,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -172,6 +176,57 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_update_stats_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+static inline void
+read_stream_update_stats_stall(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->stall_count++;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +435,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -544,7 +602,8 @@ read_stream_begin_impl(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
ReadStream *stream;
size_t size;
@@ -703,6 +762,11 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
+ /* set capacity */
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = max_pinned_buffers;
+
/*
* Skip the initial ramp-up phase if the caller says we're going to be
* reading the whole relation. This way we start out assuming we'll be
@@ -742,7 +806,8 @@ read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
return read_stream_begin_impl(flags,
strategy,
@@ -752,7 +817,8 @@ read_stream_begin_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ stats);
}
/*
@@ -777,7 +843,8 @@ read_stream_begin_smgr_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ NULL);
}
/*
@@ -851,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
@@ -860,6 +928,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_update_stats_prefetch(stream);
}
else
{
@@ -916,12 +990,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we need to wait for IO */
+ if (needed_wait)
+ read_stream_update_stats_stall(stream);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1060,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_update_stats_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index ce340c076f8..32ef27c546f 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -16,6 +16,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
+#include "executor/instrument_node.h"
#include "nodes/tidbitmap.h"
#include "port/atomics.h"
#include "storage/relfilelocator.h"
@@ -62,6 +63,11 @@ typedef struct TableScanDescData
*/
uint32 rs_flags;
+ /*
+ * IO statistics for the scan, optionally filled by the table AM.
+ */
+ IOStats rs_iostats;
+
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
} TableScanDescData;
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 8847d7f94fa..e6c714aa662 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -40,6 +40,47 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+#define ACCUMULATE_IO_STATS(dst, src) \
+ do { \
+ (dst)->prefetch_count += (src)->prefetch_count; \
+ (dst)->distance_sum += (src)->distance_sum; \
+ if ((src)->distance_max > (dst)->distance_max) \
+ (dst)->distance_max = (src)->distance_max; \
+ if ((src)->distance_capacity > (dst)->distance_capacity) \
+ (dst)->distance_capacity = (src)->distance_capacity; \
+ (dst)->stall_count += (src)->stall_count; \
+ (dst)->io_count += (src)->io_count; \
+ (dst)->io_nblocks += (src)->io_nblocks; \
+ (dst)->io_in_progress += (src)->io_in_progress; \
+ } while (0)
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
@@ -71,6 +112,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ IOStats io;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..b56bf925fc1 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -87,7 +88,8 @@ extern ReadStream *read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size);
+ size_t per_buffer_data_size,
+ IOStats *stat);
extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_data);
extern BlockNumber read_stream_next_block(ReadStream *stream,
BufferAccessStrategy *strategy);
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..bb9fbe4a407 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -68,7 +68,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +76,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -93,7 +93,7 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------
<explain xmlns="http://www.postgresql.org/N/explain"> +
@@ -144,7 +144,7 @@ select explain_filter('explain (analyze, buffers, format xml) select * from int8
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
explain_filter
-------------------------------
- Plan: +
@@ -396,7 +396,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -744,7 +744,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -772,7 +772,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -782,7 +782,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -795,7 +795,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -807,7 +807,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 00c30b91459..425aed4d749 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1012be91689 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..0124c37ef15 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -63,11 +63,11 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
@@ -102,7 +102,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -174,17 +174,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index 8d1cdd6990c..d1f80e18b41 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
--
2.53.0
[text/x-patch] v4-0003-show-prefetch-stats-for-SeqScan.patch (37.6K, ../../2551c884-af06-4663-83d2-cd5688c3ae12@vondra.me/4-v4-0003-show-prefetch-stats-for-SeqScan.patch)
download | inline diff:
From 27b92895be48cbe8bd90d78efb7586d784aa52ed Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Thu, 19 Mar 2026 14:29:52 +0100
Subject: [PATCH v4 3/4] show prefetch stats for SeqScan
- enable show_scan_io_usage for SeqScan
- add infrastructure to allocate/collect instrumentation from parallel workers
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/backend/commands/explain.c | 27 +++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeSeqscan.c | 110 +++++++++++++++++-
src/include/executor/instrument_node.h | 16 ++-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 1 +
src/test/isolation/expected/merge-update.out | 2 +-
src/test/isolation/specs/merge-update.spec | 5 +-
src/test/regress/expected/explain.out | 12 +-
src/test/regress/expected/partition_prune.out | 34 +++---
src/test/regress/sql/explain.sql | 12 +-
src/test/regress/sql/partition_prune.sql | 34 +++---
14 files changed, 213 insertions(+), 60 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 0f5271d476e..3f178eda913 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11937,7 +11937,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12183,7 +12183,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12227,7 +12227,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12387,7 +12387,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 49ed797e8ef..2aee5bb3fe9 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4059,7 +4059,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4134,13 +4134,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4192,7 +4192,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index d98962f50b3..07a418e50cc 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4082,6 +4082,23 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
/* Initialize counters with stats from the local process first */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->io);
+ }
+ }
+
+ break;
+ }
case T_BitmapHeapScan:
{
SharedBitmapHeapInstrumentation *sinstrument
@@ -4125,6 +4142,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
/* get instrumentation for the given worker */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->io;
+
+ break;
+ }
case T_BitmapHeapScan:
{
BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ac84af294c9..ffc708ed6be 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1119,6 +1119,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..2ecc0e3054d 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -295,6 +295,30 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->io, &node->ss.ss_currentScanDesc->rs_iostats);
+ }
+
/*
* get information from node
*/
@@ -349,10 +373,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -368,14 +405,42 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -404,9 +469,48 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+
+ /*
+ * Workers don't get the pscan_len value in scan descriptor, so use the
+ * TAM callback again. The result has to match the earlier result in
+ * ExecSeqScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index e6c714aa662..ce643ef0635 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -82,9 +82,23 @@ typedef struct IOStats
} while (0)
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information for sequential scans
* ---------------------
*/
+typedef struct SeqScanInstrumentation
+{
+ IOStats io;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 0716c5a9aed..f287f8fc813 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1644,6 +1644,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..dd9c67c4dc1 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..cf387eccefc 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -33,6 +33,9 @@ setup
FOR ln IN EXECUTE $1 LOOP
-- Ignore hash memory usage because it varies depending on the system
CONTINUE WHEN (ln ~ 'Memory Usage');
+ -- Ignore prefetch and I/O because it varies depending on the system
+ CONTINUE WHEN (ln ~ 'Prefetch');
+ CONTINUE WHEN (ln ~ 'I/O');
RETURN NEXT ln;
END LOOP;
END;
@@ -165,7 +168,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index bb9fbe4a407..97cb7206676 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers off, io off, verbose) select *
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -290,7 +290,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
explain_filter
-------------------------------------
[ +
@@ -425,7 +425,7 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
Planning Time: N.N
(1 row)
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
explain_filter
------------------------------------
[ +
@@ -528,7 +528,7 @@ set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -753,7 +753,7 @@ select explain_filter('explain (analyze,buffers off,io off,serialize) select * f
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual rows=N.N loops=N)
@@ -762,7 +762,7 @@ select explain_filter('explain (analyze,serialize text,buffers,timing off) selec
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1012be91689..6bd15f26aec 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2315,7 +2315,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
@@ -2325,7 +2325,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
@@ -2340,7 +2340,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2376,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2687,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2864,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3019,7 +3019,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3046,7 +3046,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3097,7 +3097,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3124,7 +3124,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3194,7 +3194,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3232,7 +3232,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3478,7 +3478,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
@@ -3498,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3513,7 +3513,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3535,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3550,7 +3550,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan expr_1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0124c37ef15..5ee453a0a96 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -65,7 +65,7 @@ explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
@@ -79,7 +79,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
set track_io_timing = off;
-- SETTINGS option
@@ -104,7 +104,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
select explain_filter('explain (memory) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
select explain_filter('explain (memory) execute int8_query');
@@ -144,7 +144,7 @@ set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -175,8 +175,8 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..c5968932aed 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,7 +669,7 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
--
2.53.0
[text/x-patch] v4-0004-show-prefetch-stats-for-TidRangeScan.patch (9.9K, ../../2551c884-af06-4663-83d2-cd5688c3ae12@vondra.me/5-v4-0004-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From f2843d05463986d3aa704a7188cb51d89f7f5826 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 20:55:00 +0100
Subject: [PATCH v4 4/4] show prefetch stats for TidRangeScan
- enable show_scan_io_usage for TidRangeScan
- add infrastructure to allocate/collect instrumentation from parallel
workers
---
src/backend/commands/explain.c | 28 ++++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeTidrangescan.c | 110 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 11 +++
src/include/executor/nodeTidrangescan.h | 1 +
src/include/nodes/execnodes.h | 1 +
6 files changed, 149 insertions(+), 5 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 07a418e50cc..8f4e3b07c3e 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2152,6 +2152,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4114,6 +4115,23 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->io);
+ }
+ }
+
break;
}
default:
@@ -4160,6 +4178,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
stats = &instrument->io;
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ TidRangeScanState *state = ((TidRangeScanState *) planstate);
+ SharedTidRangeScanInstrumentation *sinstrument = state->trss_sinstrument;
+ TidRangeScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->io;
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ffc708ed6be..62b26bfd18a 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1122,6 +1122,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 617713bde04..4ff6e333144 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -340,6 +340,29 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new state and zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->io, &node->ss.ss_currentScanDesc->rs_iostats);
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -433,11 +456,23 @@ void
ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->trss_pscanlen = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
- node->trss_pscanlen =
- table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->trss_pscanlen);
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -452,8 +487,18 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
- pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
+ size = MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
@@ -461,6 +506,23 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+ }
+
+ node->trss_sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -489,10 +551,48 @@ void
ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
pscan);
+
+ /*
+ * Workers don't get the trss_pscanlen value in scan descriptor, so use
+ * the TAM callback again. The result has to match the earlier result in
+ * ExecTidRangeScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecTidRangeScanRetrieveInstrumentation
+ *
+ * Transfer scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index ce643ef0635..b33b296ce03 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -99,6 +99,17 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+typedef struct TidRangeScanInstrumentation
+{
+ IOStats io;
+} TidRangeScanInstrumentation;
+
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..4f5dd74e0fc 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -27,5 +27,6 @@ extern void ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *p
extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index f287f8fc813..611a6280987 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1896,6 +1896,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-26 00:35 ` Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-26 00:35 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
Here's a slightly improved v5, with a couple minor changes over v4.
Patches 0001 and 0002 are prerequisites from other threads. 0001 was
already included in v4. 0002 is new and comes from [1], and it adds
"flags" to TAM beginscan() methods. This allows 0003 to pass a new
SO_SCAN_INSTRUMENT flag, so that e.g. heap_beginscan() can initialize
the instrumentation only when executed in EXPLAIN ANALYZE. In v4 it had
to be done every time, which did not seem quite great.
This mimics what we do for index scans in the index prefetching, which
only allocates the instrumentation when needed.
0004 and 0005 are adjusted to do this for SeqScan and TidRangeScans too.
0006 is a minor adjustment of the explain output, based on the earlier
discussion about non-text formats. It removes the explain groups (to not
add nested output), and instead makes the labels more descriptive. It
also prints the items every time, even with io_count==0.
The last change is that I added the number of I/Os, instead of printing
just the number of stalls and average I/O size. Without the count it
seemed a bit difficult to interpret. It might be possible to deduce some
of this from buffers reads, but I found it a bit unclear.
0006 should be split and merged into the earlier parts, I mostly kept it
to make those changes obvious.
regards
https://www.postgresql.org/message-id/itvgqc6vncbjsjfmrptfvkkeg5vqzhalaguya2z77t6c6ctpc3%40wsdrgbn4b...
--
Tomas Vondra
Attachments:
[text/x-patch] v5-0006-adjust-explain-output.patch (14.9K, ../../ab5b6aaf-90d1-4c35-b76b-6416385df6c6@vondra.me/2-v5-0006-adjust-explain-output.patch)
download | inline diff:
From b7f7eb1154ec292d7ef464850b98709607c6bd51 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Thu, 26 Mar 2026 01:21:20 +0100
Subject: [PATCH v5 6/6] adjust explain output
- Don't use groups in non-text formats.
- Don't condition fields based on (io_count > 0).
- Print complete labels in non-text formats.
---
src/backend/commands/explain.c | 35 ++--
src/test/regress/expected/explain.out | 220 ++++++++++++++------------
src/test/regress/sql/explain.sql | 4 +-
3 files changed, 132 insertions(+), 127 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5933993533e..36ddb5e5852 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4018,8 +4018,8 @@ print_io_usage(ExplainState *es, IOStats *stats)
if (stats->io_count > 0)
{
ExplainIndentText(es);
- appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
- stats->stall_count);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " stalls=%" PRIu64,
+ stats->io_count, stats->stall_count);
appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
(stats->io_nblocks * 1.0 / stats->io_count),
@@ -4030,30 +4030,21 @@ print_io_usage(ExplainState *es, IOStats *stats)
}
else
{
- ExplainOpenGroup("Prefetch", "Prefetch", true, es);
-
- ExplainPropertyFloat("Average Distance", NULL,
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
(stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
- ExplainPropertyInteger("Max Distance", NULL,
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
stats->distance_max, es);
- ExplainPropertyInteger("Capacity", NULL,
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
stats->distance_capacity, es);
- ExplainCloseGroup("Prefetch", "Prefetch", true, es);
-
- if (stats->io_count > 0)
- {
- ExplainOpenGroup("I/O", "I/O", true, es);
-
- ExplainPropertyUInteger("Stalls", NULL,
- stats->stall_count, es);
- ExplainPropertyFloat("Average IO Size", NULL,
- (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
- ExplainPropertyFloat("Average IOs In Progress", NULL,
- (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
-
- ExplainCloseGroup("I/O", "I/O", true, es);
- }
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Stalls", NULL,
+ stats->stall_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
}
}
}
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 97cb7206676..3fa6d838545 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,112 +93,126 @@ select explain_filter('explain (analyze, buffers, io off, format text) select *
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
+ explain_filter
+------------------------------------------------------------------
+ <explain xmlns="http://www.postgresql.org/N/explain"> +
+ <Query> +
+ <Plan> +
+ <Node-Type>Seq Scan</Node-Type> +
+ <Parallel-Aware>false</Parallel-Aware> +
+ <Async-Capable>false</Async-Capable> +
+ <Relation-Name>int8_tbl</Relation-Name> +
+ <Alias>i8</Alias> +
+ <Startup-Cost>N.N</Startup-Cost> +
+ <Total-Cost>N.N</Total-Cost> +
+ <Plan-Rows>N</Plan-Rows> +
+ <Plan-Width>N</Plan-Width> +
+ <Actual-Startup-Time>N.N</Actual-Startup-Time> +
+ <Actual-Total-Time>N.N</Actual-Total-Time> +
+ <Actual-Rows>N.N</Actual-Rows> +
+ <Actual-Loops>N</Actual-Loops> +
+ <Disabled>false</Disabled> +
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>+
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance> +
+ <Prefetch-Capacity>N</Prefetch-Capacity> +
+ <I-O-Count>N</I-O-Count> +
+ <I-O-Stalls>N</I-O-Stalls> +
+ <Average-I-O-Size>N.N</Average-I-O-Size> +
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress> +
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
+ <Shared-Read-Blocks>N</Shared-Read-Blocks> +
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks> +
+ <Shared-Written-Blocks>N</Shared-Written-Blocks> +
+ <Local-Hit-Blocks>N</Local-Hit-Blocks> +
+ <Local-Read-Blocks>N</Local-Read-Blocks> +
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
+ <Local-Written-Blocks>N</Local-Written-Blocks> +
+ <Temp-Read-Blocks>N</Temp-Read-Blocks> +
+ <Temp-Written-Blocks>N</Temp-Written-Blocks> +
+ </Plan> +
+ <Planning> +
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
+ <Shared-Read-Blocks>N</Shared-Read-Blocks> +
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks> +
+ <Shared-Written-Blocks>N</Shared-Written-Blocks> +
+ <Local-Hit-Blocks>N</Local-Hit-Blocks> +
+ <Local-Read-Blocks>N</Local-Read-Blocks> +
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
+ <Local-Written-Blocks>N</Local-Written-Blocks> +
+ <Temp-Read-Blocks>N</Temp-Read-Blocks> +
+ <Temp-Written-Blocks>N</Temp-Written-Blocks> +
+ </Planning> +
+ <Planning-Time>N.N</Planning-Time> +
+ <Triggers> +
+ </Triggers> +
+ <Execution-Time>N.N</Execution-Time> +
+ </Query> +
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
+ explain_filter
+------------------------------------
+ - Plan: +
+ Node Type: "Seq Scan" +
+ Parallel Aware: false +
+ Async Capable: false +
+ Relation Name: "int8_tbl" +
+ Alias: "i8" +
+ Startup Cost: N.N +
+ Total Cost: N.N +
+ Plan Rows: N +
+ Plan Width: N +
+ Actual Startup Time: N.N +
+ Actual Total Time: N.N +
+ Actual Rows: N.N +
+ Actual Loops: N +
+ Disabled: false +
+ Average Prefetch Distance: N.N+
+ Max Prefetch Distance: N +
+ Prefetch Capacity: N +
+ I/O Count: N +
+ I/O Stalls: N +
+ Average I/O Size: N.N +
+ Average I/Os In Progress: N.N +
+ Shared Hit Blocks: N +
+ Shared Read Blocks: N +
+ Shared Dirtied Blocks: N +
+ Shared Written Blocks: N +
+ Local Hit Blocks: N +
+ Local Read Blocks: N +
+ Local Dirtied Blocks: N +
+ Local Written Blocks: N +
+ Temp Read Blocks: N +
+ Temp Written Blocks: N +
+ Planning: +
+ Shared Hit Blocks: N +
+ Shared Read Blocks: N +
+ Shared Dirtied Blocks: N +
+ Shared Written Blocks: N +
+ Local Hit Blocks: N +
+ Local Read Blocks: N +
+ Local Dirtied Blocks: N +
+ Local Written Blocks: N +
+ Temp Read Blocks: N +
+ Temp Written Blocks: N +
+ Planning Time: N.N +
+ Triggers: +
+ Serialization: +
+ Time: N.N +
+ Output Volume: N +
+ Format: "text" +
+ Shared Hit Blocks: N +
+ Shared Read Blocks: N +
+ Shared Dirtied Blocks: N +
+ Shared Written Blocks: N +
+ Local Hit Blocks: N +
+ Local Read Blocks: N +
+ Local Dirtied Blocks: N +
+ Local Written Blocks: N +
+ Temp Read Blocks: N +
+ Temp Written Blocks: N +
Execution Time: N.N
(1 row)
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 5ee453a0a96..7acd52da0db 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,8 +66,8 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
--
2.53.0
[text/x-patch] v5-0005-show-prefetch-stats-for-TidRangeScan.patch (10.5K, ../../ab5b6aaf-90d1-4c35-b76b-6416385df6c6@vondra.me/3-v5-0005-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From 3f53baf68d9dabf7728829055c6d207b2eadc0f1 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 20:55:00 +0100
Subject: [PATCH v5 5/6] show prefetch stats for TidRangeScan
- enable show_scan_io_usage for TidRangeScan
- add infrastructure to allocate/collect instrumentation from parallel
workers
---
src/backend/commands/explain.c | 31 +++++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeTidrangescan.c | 118 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 11 +++
src/include/executor/nodeTidrangescan.h | 1 +
src/include/nodes/execnodes.h | 1 +
6 files changed, 157 insertions(+), 8 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index fe137978e0c..5933993533e 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2152,6 +2152,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4117,6 +4118,26 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->stats.io);
+ }
+ }
+
break;
}
default:
@@ -4163,6 +4184,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
stats = &instrument->stats.io;
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ TidRangeScanState *state = ((TidRangeScanState *) planstate);
+ SharedTidRangeScanInstrumentation *sinstrument = state->trss_sinstrument;
+ TidRangeScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ffc708ed6be..62b26bfd18a 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1122,6 +1122,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index bacd7aa5bc4..d0bf98654c8 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -243,7 +243,7 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
- 0,
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid);
@@ -341,6 +341,29 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new state and zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->stats.io, &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -434,11 +457,23 @@ void
ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->trss_pscanlen = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
- node->trss_pscanlen =
- table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->trss_pscanlen);
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -453,15 +488,43 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
- pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
+ size = MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- 0, pscan);
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
+ pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+ }
+
+ node->trss_sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -490,10 +553,49 @@ void
ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- 0, pscan);
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
+ pscan);
+
+ /*
+ * Workers don't get the trss_pscanlen value in scan descriptor, so use
+ * the TAM callback again. The result has to match the earlier result in
+ * ExecTidRangeScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecTidRangeScanRetrieveInstrumentation
+ *
+ * Transfer scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index f27cef56d07..5e7237a241b 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -104,6 +104,17 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..4f5dd74e0fc 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -27,5 +27,6 @@ extern void ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *p
extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 90da865518c..903f8c1160c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1896,6 +1896,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
--
2.53.0
[text/x-patch] v5-0004-show-prefetch-stats-for-SeqScan.patch (38.3K, ../../ab5b6aaf-90d1-4c35-b76b-6416385df6c6@vondra.me/4-v5-0004-show-prefetch-stats-for-SeqScan.patch)
download | inline diff:
From 3974edba4a5197a3beec4546fba335eff938bee0 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Mon, 23 Mar 2026 11:29:15 +0100
Subject: [PATCH v5 4/6] show prefetch stats for SeqScan
- enable show_scan_io_usage for SeqScan
- add infrastructure to allocate/collect instrumentation from parallel workers
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/backend/commands/explain.c | 30 +++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeSeqscan.c | 122 +++++++++++++++++-
src/include/executor/instrument_node.h | 15 ++-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 1 +
src/test/isolation/expected/merge-update.out | 2 +-
src/test/isolation/specs/merge-update.spec | 5 +-
src/test/regress/expected/explain.out | 12 +-
src/test/regress/expected/partition_prune.out | 34 ++---
src/test/regress/sql/explain.sql | 12 +-
src/test/regress/sql/partition_prune.sql | 34 ++---
14 files changed, 223 insertions(+), 64 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 0f5271d476e..3f178eda913 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11937,7 +11937,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12183,7 +12183,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12227,7 +12227,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12387,7 +12387,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 49ed797e8ef..2aee5bb3fe9 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4059,7 +4059,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4134,13 +4134,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4192,7 +4192,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 04f21187b4c..fe137978e0c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4079,6 +4079,26 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
/* Initialize counters with stats from the local process first */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->stats.io);
+ }
+ }
+
+ break;
+ }
case T_BitmapHeapScan:
{
SharedBitmapHeapInstrumentation *sinstrument
@@ -4125,6 +4145,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
/* get instrumentation for the given worker */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
+ break;
+ }
case T_BitmapHeapScan:
{
BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ac84af294c9..ffc708ed6be 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1119,6 +1119,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 376e877e87c..95297719975 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -70,8 +70,8 @@ SeqNext(SeqScanState *node)
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
- 0, estate->es_snapshot,
- 0, NULL);
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
+ estate->es_snapshot, 0, NULL);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -295,6 +295,30 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->stats.io, &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+
/*
* get information from node
*/
@@ -349,10 +373,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -368,14 +405,44 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pscan = shm_toc_allocate(pcxt->toc, size);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
+ table_beginscan_parallel(node->ss.ss_currentRelation,
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
+ pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -404,9 +471,50 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ Size size;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
+ table_beginscan_parallel(node->ss.ss_currentRelation,
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
+ pscan);
+
+ /*
+ * Workers don't get the pscan_len value in scan descriptor, so use the
+ * TAM callback again. The result has to match the earlier result in
+ * ExecSeqScanEstimate.
+ */
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(size);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 75b09c4e256..f27cef56d07 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -87,9 +87,22 @@ typedef struct TableScanInstrumentation
} while (0)
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information for sequential scans
* ---------------------
*/
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
typedef struct IndexScanInstrumentation
{
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 684e398f824..90da865518c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1644,6 +1644,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..dd9c67c4dc1 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..cf387eccefc 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -33,6 +33,9 @@ setup
FOR ln IN EXECUTE $1 LOOP
-- Ignore hash memory usage because it varies depending on the system
CONTINUE WHEN (ln ~ 'Memory Usage');
+ -- Ignore prefetch and I/O because it varies depending on the system
+ CONTINUE WHEN (ln ~ 'Prefetch');
+ CONTINUE WHEN (ln ~ 'I/O');
RETURN NEXT ln;
END LOOP;
END;
@@ -165,7 +168,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index bb9fbe4a407..97cb7206676 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers off, io off, verbose) select *
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -290,7 +290,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
explain_filter
-------------------------------------
[ +
@@ -425,7 +425,7 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
Planning Time: N.N
(1 row)
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
explain_filter
------------------------------------
[ +
@@ -528,7 +528,7 @@ set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -753,7 +753,7 @@ select explain_filter('explain (analyze,buffers off,io off,serialize) select * f
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual rows=N.N loops=N)
@@ -762,7 +762,7 @@ select explain_filter('explain (analyze,serialize text,buffers,timing off) selec
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1012be91689..6bd15f26aec 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2315,7 +2315,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
@@ -2325,7 +2325,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
@@ -2340,7 +2340,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2376,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2687,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2864,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3019,7 +3019,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3046,7 +3046,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3097,7 +3097,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3124,7 +3124,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3194,7 +3194,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3232,7 +3232,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3478,7 +3478,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
@@ -3498,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3513,7 +3513,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3535,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3550,7 +3550,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan expr_1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 0124c37ef15..5ee453a0a96 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -65,7 +65,7 @@ explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
@@ -79,7 +79,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
set track_io_timing = off;
-- SETTINGS option
@@ -104,7 +104,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
select explain_filter('explain (memory) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
select explain_filter('explain (memory) execute int8_query');
@@ -144,7 +144,7 @@ set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -175,8 +175,8 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..c5968932aed 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,7 +669,7 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
--
2.53.0
[text/x-patch] v5-0003-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch (71.5K, ../../ab5b6aaf-90d1-4c35-b76b-6416385df6c6@vondra.me/5-v5-0003-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch)
download | inline diff:
From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Thu, 19 Mar 2026 22:25:09 +0100
Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
This adds details about AIO / prefetch for executor nodes using a
ReadStream. Right now this applies only to BitmapHeapScan, because
that's the only scan node using a ReadStream and collecting
instrumentation from workers.
The stats are collected by the ReadStream unconditionally, i.e. without
EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
integer counters), it's not worth gating this by a flag.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) so that the executor can access it and use it for explain.
The collected stats are designed for ReadStream, but is meant to be
reasonably generic in case a TAM manages I/Os in different ways.
In EXPLAIN, the statistic is triggered by a new option "IO", enabled by
default. This mimics what we do for BUFFERS, which seems like the
closest EXPLAIN option.
---
contrib/amcheck/verify_heapam.c | 3 +-
contrib/bloom/blscan.c | 3 +-
contrib/bloom/blvacuum.c | 6 +-
contrib/pg_prewarm/autoprewarm.c | 3 +-
contrib/pg_prewarm/pg_prewarm.c | 3 +-
contrib/pg_visibility/pg_visibility.c | 6 +-
contrib/pgstattuple/pgstatapprox.c | 3 +-
contrib/pgstattuple/pgstatindex.c | 6 +-
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/brin/brin.c | 3 +-
src/backend/access/gin/ginvacuum.c | 3 +-
src/backend/access/gist/gistvacuum.c | 3 +-
src/backend/access/hash/hash.c | 3 +-
src/backend/access/heap/heapam.c | 13 +-
src/backend/access/heap/vacuumlazy.c | 6 +-
src/backend/access/nbtree/nbtree.c | 3 +-
src/backend/access/spgist/spgvacuum.c | 3 +-
src/backend/commands/analyze.c | 3 +-
src/backend/commands/explain.c | 162 +++++++++++++++++-
src/backend/commands/explain_state.c | 15 ++
src/backend/executor/nodeBitmapHeapscan.c | 6 +-
src/backend/storage/aio/read_stream.c | 91 +++++++++-
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 48 ++++++
src/include/storage/read_stream.h | 4 +-
.../modules/index/expected/killtuples.out | 44 ++---
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/regress/expected/explain.out | 20 +--
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 2 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 20 +--
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
43 files changed, 452 insertions(+), 92 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 20ff58aa782..5cfe8e24615 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -477,7 +477,8 @@ verify_heapam(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
stream_cb,
stream_data,
- 0);
+ 0,
+ NULL);
while ((ctx.buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
{
diff --git a/contrib/bloom/blscan.c b/contrib/bloom/blscan.c
index 1a0e42021ec..9bdc4a95c18 100644
--- a/contrib/bloom/blscan.c
+++ b/contrib/bloom/blscan.c
@@ -138,7 +138,8 @@ blgetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
index 6beb1c20ebb..587757c1e18 100644
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -71,7 +71,8 @@ blbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
@@ -224,7 +225,8 @@ blvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index ba0bc8e6d4a..4c5448f177a 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -628,7 +628,8 @@ autoprewarm_database_main(Datum main_arg)
p.forknum,
apw_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/*
* Loop until we've prewarmed all the blocks from this fork. The
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index c2716086693..e4541f46777 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -256,7 +256,8 @@ pg_prewarm(PG_FUNCTION_ARGS)
forkNumber,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (block = first_block; block <= last_block; ++block)
{
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dfab0b64cf5..f09bb68624f 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -519,7 +519,8 @@ collect_visibility_data(Oid relid, bool include_pd)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
}
for (blkno = 0; blkno < nblocks; ++blkno)
@@ -739,7 +740,8 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen)
MAIN_FORKNUM,
collect_corrupt_items_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/* Loop over every block in the relation. */
while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 21e0b50fb4b..a926a1f1168 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -146,7 +146,8 @@ statapprox_heap(Relation rel, output_type *stat)
MAIN_FORKNUM,
statapprox_heap_read_stream_next,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 3a3f2637bd9..0de33558c47 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -296,7 +296,8 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
@@ -685,7 +686,8 @@ pgstathashindex(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..4951b624893 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>TRUE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 79a79bea1c6..cfb3348b682 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2192,7 +2192,8 @@ brin_vacuum_scan(Relation idxrel, BufferAccessStrategy strategy)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* Scan the index in physical order, and clean up any possible mess in
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 840543eb664..484360a9d6d 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -799,7 +799,8 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = GIN_ROOT_BLKNO; blkno < npages; blkno++)
{
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 686a0418054..e4f578b2de0 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -223,7 +223,8 @@ gistvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 8d8cd30dc38..acd77fd6093 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -547,7 +547,8 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
hash_bulkdelete_read_stream_cb,
&stream_private,
- 0);
+ 0,
+ NULL);
bucket_loop:
while (cur_bucket <= cur_maxbucket)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 044f385e477..507e1bef2ec 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1198,6 +1198,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1267,6 +1268,12 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_read_stream = NULL;
+ /* allocate instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ else
+ scan->rs_base.rs_instrument = NULL;
+
/*
* Set up a read stream for sequential scans and TID range scans. This
* should be done after initscan() because initscan() allocates the
@@ -1296,7 +1303,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
cb,
scan,
- 0);
+ 0,
+ &scan->rs_base.rs_instrument->io);
}
else if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
{
@@ -1307,7 +1315,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
bitmapheap_stream_read_next,
scan,
- sizeof(TBMIterateResult));
+ sizeof(TBMIterateResult),
+ &scan->rs_base.rs_instrument->io);
}
scan->rs_vmbuffer = InvalidBuffer;
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index f698c2d899b..aa2d4a7931b 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1304,7 +1304,8 @@ lazy_scan_heap(LVRelState *vacrel)
MAIN_FORKNUM,
heap_vac_scan_next_block,
vacrel,
- sizeof(bool));
+ sizeof(bool),
+ NULL);
while (true)
{
@@ -2663,7 +2664,8 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
MAIN_FORKNUM,
vacuum_reap_lp_read_stream_next,
iter,
- sizeof(TidStoreIterResult));
+ sizeof(TidStoreIterResult),
+ NULL);
while (true)
{
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index 6d870e4ebe7..98b7bc5c1b8 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -1326,7 +1326,8 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index c461f8dc02d..d8da3ebf43a 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -835,7 +835,8 @@ spgvacuumscan(spgBulkDeleteState *bds)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* The outer loop iterates over all index pages except the metapage, in
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index eeed91be266..a2cf176a485 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -1251,7 +1251,8 @@ acquire_sample_rows(Relation onerel, int elevel,
MAIN_FORKNUM,
block_sampling_read_stream_next,
&bs,
- 0);
+ 0,
+ NULL);
/* Outer loop over blocks to sample */
while (table_scan_analyze_next_block(scan, stream))
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4b70166b0e..04f21187b4c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,8 @@
*/
#include "postgres.h"
+#include "access/genam.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +141,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
+static void show_io_usage(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2016,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2035,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2297,8 +2306,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->walusage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
@@ -2315,6 +2324,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+ if (es->io)
+ show_io_usage(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3985,6 +3997,152 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+ appendStringInfoChar(es->str, '\n');
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
+ stats->stall_count);
+
+ appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+
+ appendStringInfoChar(es->str, '\n');
+ }
+ }
+ else
+ {
+ ExplainOpenGroup("Prefetch", "Prefetch", true, es);
+
+ ExplainPropertyFloat("Average Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainCloseGroup("Prefetch", "Prefetch", true, es);
+
+ if (stats->io_count > 0)
+ {
+ ExplainOpenGroup("I/O", "I/O", true, es);
+
+ ExplainPropertyUInteger("Stalls", NULL,
+ stats->stall_count, es);
+ ExplainPropertyFloat("Average IO Size", NULL,
+ (stats->io_nblocks * 1.0 / stats->io_count), 3, es);
+ ExplainPropertyFloat("Average IOs In Progress", NULL,
+ (stats->io_in_progress * 1.0 / stats->io_count), 3, es);
+
+ ExplainCloseGroup("I/O", "I/O", true, es);
+ }
+ }
+ }
+}
+
+/*
+ * show_scan_io_usage
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started, no prefetch stats */
+ if (!(planstate && planstate->ss_currentScanDesc))
+ return;
+
+ /* Initialize counters with stats from the local process first */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+ ACCUMULATE_IO_STATS(&stats, &winstrument->stats.io);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
+/*
+ * show_io_usage
+ * show info about I/O prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_io_usage(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ IOStats *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -159,6 +160,11 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ io_set = true;
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -179,12 +185,21 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e2c1b7467b..d181fd7b8ff 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -104,6 +104,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
TBMIterator tbmiterator = {0};
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
+ EState *estate = node->ss.ps.state;
if (!pstate)
{
@@ -146,7 +147,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
{
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
- 0,
+ (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
node->ss.ps.state->es_snapshot,
0,
NULL);
@@ -329,6 +330,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ ACCUMULATE_IO_STATS(&si->stats.io, &node->ss.ss_currentScanDesc->rs_instrument->io);
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74ac..56b0b8e7fb5 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -107,6 +108,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -172,6 +176,57 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_update_stats_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+static inline void
+read_stream_update_stats_stall(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->stall_count++;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +435,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -544,7 +602,8 @@ read_stream_begin_impl(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
ReadStream *stream;
size_t size;
@@ -703,6 +762,11 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
+ /* set capacity */
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = max_pinned_buffers;
+
/*
* Skip the initial ramp-up phase if the caller says we're going to be
* reading the whole relation. This way we start out assuming we'll be
@@ -742,7 +806,8 @@ read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
return read_stream_begin_impl(flags,
strategy,
@@ -752,7 +817,8 @@ read_stream_begin_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ stats);
}
/*
@@ -777,7 +843,8 @@ read_stream_begin_smgr_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ NULL);
}
/*
@@ -851,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_update_stats_prefetch(stream);
return buffer;
}
@@ -860,6 +928,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* update I/O stats */
+ read_stream_update_stats_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_update_stats_prefetch(stream);
}
else
{
@@ -916,12 +990,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we need to wait for IO */
+ if (needed_wait)
+ read_stream_update_stats_stall(stream);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1060,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_update_stats_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 80ea0b437d1..c5b3cd5585c 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -25,6 +25,8 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -64,6 +66,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e1f90f2b6a7..c4dd07214db 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,9 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 10,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 8847d7f94fa..75b09c4e256 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -40,10 +40,57 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of stalled reads (waiting for I/O) */
+ uint64 stall_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+#define ACCUMULATE_IO_STATS(dst, src) \
+ do { \
+ (dst)->prefetch_count += (src)->prefetch_count; \
+ (dst)->distance_sum += (src)->distance_sum; \
+ if ((src)->distance_max > (dst)->distance_max) \
+ (dst)->distance_max = (src)->distance_max; \
+ if ((src)->distance_capacity > (dst)->distance_capacity) \
+ (dst)->distance_capacity = (src)->distance_capacity; \
+ (dst)->stall_count += (src)->stall_count; \
+ (dst)->io_count += (src)->io_count; \
+ (dst)->io_nblocks += (src)->io_nblocks; \
+ (dst)->io_in_progress += (src)->io_in_progress; \
+ } while (0)
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
*/
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +118,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..b56bf925fc1 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -87,7 +88,8 @@ extern ReadStream *read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size);
+ size_t per_buffer_data_size,
+ IOStats *stat);
extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_data);
extern BlockNumber read_stream_next_block(ReadStream *stream,
BufferAccessStrategy *strategy);
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..bb9fbe4a407 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -68,7 +68,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +76,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -93,7 +93,7 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------
<explain xmlns="http://www.postgresql.org/N/explain"> +
@@ -144,7 +144,7 @@ select explain_filter('explain (analyze, buffers, format xml) select * from int8
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
explain_filter
-------------------------------
- Plan: +
@@ -396,7 +396,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -744,7 +744,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -772,7 +772,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -782,7 +782,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -795,7 +795,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -807,7 +807,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 218972dfab8..950c0fbae69 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1012be91689 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..0124c37ef15 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -63,11 +63,11 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io off, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
@@ -102,7 +102,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -174,17 +174,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index e39bbb65391..532df97ee90 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
--
2.53.0
[text/x-patch] v5-0002-Thread-flags-through-begin-scan-APIs.patch (32.8K, ../../ab5b6aaf-90d1-4c35-b76b-6416385df6c6@vondra.me/6-v5-0002-Thread-flags-through-begin-scan-APIs.patch)
download | inline diff:
From 163a2b04b1e4d89b3018198102e43892b34521d9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 19 Mar 2026 17:05:55 -0400
Subject: [PATCH v5 2/6] Thread flags through begin-scan APIs
Add a user-settable flags parameter to the table_beginscan_* wrappers,
index_beginscan(), table_index_fetch_begin(), and the table
AM callback index_fetch_begin(). This allows users to pass additional
context to be used when building the scan descriptors.
For index scans, a new uint32 flags field is added to
IndexFetchTableData, and the heap AM stores the caller-provided flags
there in heapam_index_fetch_begin().
This introduces an extension point for follow-up work to pass
per-scan information (such as whether the relation is read-only for the
current query) from the executor to the AM layer.
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/F5CDD1B5-628C-44A1-9F85-3958C626F6A9%40gmail.com
---
contrib/pgrowlocks/pgrowlocks.c | 2 +-
src/backend/access/brin/brin.c | 1 +
src/backend/access/gin/gininsert.c | 1 +
src/backend/access/heap/heapam_handler.c | 9 ++-
src/backend/access/index/genam.c | 2 +
src/backend/access/index/indexam.c | 6 +-
src/backend/access/nbtree/nbtsort.c | 2 +-
src/backend/access/table/tableam.c | 19 ++++---
src/backend/commands/constraint.c | 2 +-
src/backend/commands/copyto.c | 2 +-
src/backend/commands/tablecmds.c | 9 ++-
src/backend/commands/typecmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +-
src/backend/executor/execReplication.c | 12 ++--
src/backend/executor/nodeBitmapHeapscan.c | 1 +
src/backend/executor/nodeIndexonlyscan.c | 3 +
src/backend/executor/nodeIndexscan.c | 4 ++
src/backend/executor/nodeSamplescan.c | 1 +
src/backend/executor/nodeSeqscan.c | 6 +-
src/backend/executor/nodeTidrangescan.c | 5 +-
src/backend/partitioning/partbounds.c | 2 +-
src/backend/utils/adt/selfuncs.c | 1 +
src/include/access/genam.h | 2 +
src/include/access/heapam.h | 5 +-
src/include/access/relscan.h | 1 +
src/include/access/tableam.h | 68 +++++++++++++++--------
26 files changed, 111 insertions(+), 62 deletions(-)
diff --git a/contrib/pgrowlocks/pgrowlocks.c b/contrib/pgrowlocks/pgrowlocks.c
index ff3692c87c4..95fad61fa9e 100644
--- a/contrib/pgrowlocks/pgrowlocks.c
+++ b/contrib/pgrowlocks/pgrowlocks.c
@@ -115,7 +115,7 @@ pgrowlocks(PG_FUNCTION_ARGS)
RelationGetRelationName(rel));
/* Scan the relation */
- scan = table_beginscan(rel, GetActiveSnapshot(), 0, NULL);
+ scan = table_beginscan(rel, 0, GetActiveSnapshot(), 0, NULL);
hscan = (HeapScanDesc) scan;
attinmeta = TupleDescGetAttInMetadata(rsinfo->setDesc);
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 2a0f8c8e3b8..79a79bea1c6 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2844,6 +2844,7 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
indexInfo->ii_Concurrent = brinshared->isconcurrent;
scan = table_beginscan_parallel(heap,
+ 0,
ParallelTableScanFromBrinShared(brinshared));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
diff --git a/src/backend/access/gin/gininsert.c b/src/backend/access/gin/gininsert.c
index e54782d9dd8..32167d03137 100644
--- a/src/backend/access/gin/gininsert.c
+++ b/src/backend/access/gin/gininsert.c
@@ -2068,6 +2068,7 @@ _gin_parallel_scan_and_build(GinBuildState *state,
indexInfo->ii_Concurrent = ginshared->isconcurrent;
scan = table_beginscan_parallel(heap,
+ 0,
ParallelTableScanFromGinBuildShared(ginshared));
reltuples = table_index_build_scan(heap, index, indexInfo, true, progress,
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d40878928e1..951273a4d7f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -81,11 +81,12 @@ heapam_slot_callbacks(Relation relation)
*/
static IndexFetchTableData *
-heapam_index_fetch_begin(Relation rel)
+heapam_index_fetch_begin(Relation rel, uint32 flags)
{
IndexFetchHeapData *hscan = palloc0_object(IndexFetchHeapData);
hscan->xs_base.rel = rel;
+ hscan->xs_base.flags = flags;
hscan->xs_cbuf = InvalidBuffer;
hscan->xs_vmbuffer = InvalidBuffer;
@@ -763,7 +764,9 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
tableScan = NULL;
heapScan = NULL;
- indexScan = index_beginscan(OldHeap, OldIndex, SnapshotAny, NULL, 0, 0);
+ indexScan = index_beginscan(OldHeap, OldIndex,
+ 0, /* flags */
+ SnapshotAny, NULL, 0, 0);
index_rescan(indexScan, NULL, 0, NULL, 0);
}
else
@@ -772,7 +775,7 @@ heapam_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_SEQ_SCAN_HEAP);
- tableScan = table_beginscan(OldHeap, SnapshotAny, 0, (ScanKey) NULL);
+ tableScan = table_beginscan(OldHeap, 0, SnapshotAny, 0, (ScanKey) NULL);
heapScan = (HeapScanDesc) tableScan;
indexScan = NULL;
diff --git a/src/backend/access/index/genam.c b/src/backend/access/index/genam.c
index 5e89b86a62c..b099d956e41 100644
--- a/src/backend/access/index/genam.c
+++ b/src/backend/access/index/genam.c
@@ -455,6 +455,7 @@ systable_beginscan(Relation heapRelation,
}
sysscan->iscan = index_beginscan(heapRelation, irel,
+ 0, /* flags */
snapshot, NULL, nkeys, 0);
index_rescan(sysscan->iscan, idxkey, nkeys, NULL, 0);
sysscan->scan = NULL;
@@ -716,6 +717,7 @@ systable_beginscan_ordered(Relation heapRelation,
bsysscan = true;
sysscan->iscan = index_beginscan(heapRelation, indexRelation,
+ 0, /* flags */
snapshot, NULL, nkeys, 0);
index_rescan(sysscan->iscan, idxkey, nkeys, NULL, 0);
sysscan->scan = NULL;
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index fbfc33159eb..ae754503007 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -256,6 +256,7 @@ index_insert_cleanup(Relation indexRelation,
IndexScanDesc
index_beginscan(Relation heapRelation,
Relation indexRelation,
+ uint32 flags,
Snapshot snapshot,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys)
@@ -285,7 +286,7 @@ index_beginscan(Relation heapRelation,
scan->instrument = instrument;
/* prepare to fetch index matches from table */
- scan->xs_heapfetch = table_index_fetch_begin(heapRelation);
+ scan->xs_heapfetch = table_index_fetch_begin(heapRelation, flags);
return scan;
}
@@ -592,6 +593,7 @@ index_parallelrescan(IndexScanDesc scan)
*/
IndexScanDesc
index_beginscan_parallel(Relation heaprel, Relation indexrel,
+ uint32 flags,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys,
ParallelIndexScanDesc pscan)
@@ -616,7 +618,7 @@ index_beginscan_parallel(Relation heaprel, Relation indexrel,
scan->instrument = instrument;
/* prepare to fetch index matches from table */
- scan->xs_heapfetch = table_index_fetch_begin(heaprel);
+ scan->xs_heapfetch = table_index_fetch_begin(heaprel, flags);
return scan;
}
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index 47a9bda30c9..98e9410c579 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -1928,7 +1928,7 @@ _bt_parallel_scan_and_sort(BTSpool *btspool, BTSpool *btspool2,
indexInfo = BuildIndexInfo(btspool->index);
indexInfo->ii_Concurrent = btshared->isconcurrent;
scan = table_beginscan_parallel(btspool->heap,
- ParallelTableScanFromBTShared(btshared));
+ 0, ParallelTableScanFromBTShared(btshared));
reltuples = table_index_build_scan(btspool->heap, btspool->index, indexInfo,
true, progress, _bt_build_callback,
&buildstate, scan);
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index dfda1af412e..32bd3fdb7a5 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -118,7 +118,7 @@ table_beginscan_catalog(Relation relation, int nkeys, ScanKeyData *key)
Snapshot snapshot = RegisterSnapshot(GetCatalogSnapshot(relid));
return table_beginscan_common(relation, snapshot, nkeys, key,
- NULL, flags);
+ NULL, flags, 0);
}
@@ -163,10 +163,10 @@ table_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan,
}
TableScanDesc
-table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan)
+table_beginscan_parallel(Relation relation, uint32 flags, ParallelTableScanDesc pscan)
{
Snapshot snapshot;
- uint32 flags = SO_TYPE_SEQSCAN |
+ uint32 internal_flags = SO_TYPE_SEQSCAN |
SO_ALLOW_STRAT | SO_ALLOW_SYNC | SO_ALLOW_PAGEMODE;
Assert(RelFileLocatorEquals(relation->rd_locator, pscan->phs_locator));
@@ -176,7 +176,7 @@ table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan)
/* Snapshot was serialized -- restore it */
snapshot = RestoreSnapshot((char *) pscan + pscan->phs_snapshot_off);
RegisterSnapshot(snapshot);
- flags |= SO_TEMP_SNAPSHOT;
+ internal_flags |= SO_TEMP_SNAPSHOT;
}
else
{
@@ -185,16 +185,17 @@ table_beginscan_parallel(Relation relation, ParallelTableScanDesc pscan)
}
return table_beginscan_common(relation, snapshot, 0, NULL,
- pscan, flags);
+ pscan, internal_flags, flags);
}
TableScanDesc
table_beginscan_parallel_tidrange(Relation relation,
+ uint32 flags,
ParallelTableScanDesc pscan)
{
Snapshot snapshot;
- uint32 flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
TableScanDesc sscan;
+ uint32 internal_flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
Assert(RelFileLocatorEquals(relation->rd_locator, pscan->phs_locator));
@@ -206,7 +207,7 @@ table_beginscan_parallel_tidrange(Relation relation,
/* Snapshot was serialized -- restore it */
snapshot = RestoreSnapshot((char *) pscan + pscan->phs_snapshot_off);
RegisterSnapshot(snapshot);
- flags |= SO_TEMP_SNAPSHOT;
+ internal_flags |= SO_TEMP_SNAPSHOT;
}
else
{
@@ -215,7 +216,7 @@ table_beginscan_parallel_tidrange(Relation relation,
}
sscan = table_beginscan_common(relation, snapshot, 0, NULL,
- pscan, flags);
+ pscan, internal_flags, flags);
return sscan;
}
@@ -248,7 +249,7 @@ table_index_fetch_tuple_check(Relation rel,
bool found;
slot = table_slot_create(rel, NULL);
- scan = table_index_fetch_begin(rel);
+ scan = table_index_fetch_begin(rel, 0);
found = table_index_fetch_tuple(scan, tid, snapshot, slot, &call_again,
all_dead);
table_index_fetch_end(scan);
diff --git a/src/backend/commands/constraint.c b/src/backend/commands/constraint.c
index cc11c47b6f2..37cfbd63938 100644
--- a/src/backend/commands/constraint.c
+++ b/src/backend/commands/constraint.c
@@ -106,7 +106,7 @@ unique_key_recheck(PG_FUNCTION_ARGS)
*/
tmptid = checktid;
{
- IndexFetchTableData *scan = table_index_fetch_begin(trigdata->tg_relation);
+ IndexFetchTableData *scan = table_index_fetch_begin(trigdata->tg_relation, 0);
bool call_again = false;
if (!table_index_fetch_tuple(scan, &tmptid, SnapshotSelf, slot,
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index faf62d959b4..390b4260ada 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -1336,7 +1336,7 @@ CopyRelationTo(CopyToState cstate, Relation rel, Relation root_rel, uint64 *proc
AttrMap *map = NULL;
TupleTableSlot *root_slot = NULL;
- scandesc = table_beginscan(rel, GetActiveSnapshot(), 0, NULL);
+ scandesc = table_beginscan(rel, 0, GetActiveSnapshot(), 0, NULL);
slot = table_slot_create(rel, NULL);
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index c69c12dc014..14d808671c5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -6411,7 +6411,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap)
* checking all the constraints.
*/
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(oldrel, snapshot, 0, NULL);
+ scan = table_beginscan(oldrel, 0, snapshot, 0, NULL);
/*
* Switch to per-tuple memory context and reset it for each tuple
@@ -13980,8 +13980,7 @@ validateForeignKeyConstraint(char *conname,
*/
snapshot = RegisterSnapshot(GetLatestSnapshot());
slot = table_slot_create(rel, NULL);
- scan = table_beginscan(rel, snapshot, 0, NULL);
-
+ scan = table_beginscan(rel, 0, snapshot, 0, NULL);
perTupCxt = AllocSetContextCreate(CurrentMemoryContext,
"validateForeignKeyConstraint",
ALLOCSET_SMALL_SIZES);
@@ -22882,7 +22881,7 @@ MergePartitionsMoveRows(List **wqueue, List *mergingPartitions, Relation newPart
/* Scan through the rows. */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(mergingPartition, snapshot, 0, NULL);
+ scan = table_beginscan(mergingPartition, 0, snapshot, 0, NULL);
/*
* Switch to per-tuple memory context and reset it for each tuple
@@ -23346,7 +23345,7 @@ SplitPartitionMoveRows(List **wqueue, Relation rel, Relation splitRel,
/* Scan through the rows. */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(splitRel, snapshot, 0, NULL);
+ scan = table_beginscan(splitRel, 0, snapshot, 0, NULL);
/*
* Switch to per-tuple memory context and reset it for each tuple
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 3dab6bb5a79..8c5d5e708a1 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -3185,7 +3185,7 @@ validateDomainNotNullConstraint(Oid domainoid)
/* Scan all tuples in this relation */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(testrel, snapshot, 0, NULL);
+ scan = table_beginscan(testrel, 0, snapshot, 0, NULL);
slot = table_slot_create(testrel, NULL);
while (table_scan_getnextslot(scan, ForwardScanDirection, slot))
{
@@ -3266,7 +3266,7 @@ validateDomainCheckConstraint(Oid domainoid, const char *ccbin, LOCKMODE lockmod
/* Scan all tuples in this relation */
snapshot = RegisterSnapshot(GetLatestSnapshot());
- scan = table_beginscan(testrel, snapshot, 0, NULL);
+ scan = table_beginscan(testrel, 0, snapshot, 0, NULL);
slot = table_slot_create(testrel, NULL);
while (table_scan_getnextslot(scan, ForwardScanDirection, slot))
{
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9d071e495c6..c46beedeb71 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -815,7 +815,8 @@ check_exclusion_or_unique_constraint(Relation heap, Relation index,
retry:
conflict = false;
found_self = false;
- index_scan = index_beginscan(heap, index, &DirtySnapshot, NULL, indnkeyatts, 0);
+ index_scan = index_beginscan(heap, index, 0, /* flags */
+ &DirtySnapshot, NULL, indnkeyatts, 0);
index_rescan(index_scan, scankeys, indnkeyatts, NULL, 0);
while (index_getnext_slot(index_scan, ForwardScanDirection, existing_slot))
diff --git a/src/backend/executor/execReplication.c b/src/backend/executor/execReplication.c
index 2497ee7edc5..3ef4d5d8bb2 100644
--- a/src/backend/executor/execReplication.c
+++ b/src/backend/executor/execReplication.c
@@ -205,7 +205,9 @@ RelationFindReplTupleByIndex(Relation rel, Oid idxoid,
skey_attoff = build_replindex_scan_key(skey, rel, idxrel, searchslot);
/* Start an index scan. */
- scan = index_beginscan(rel, idxrel, &snap, NULL, skey_attoff, 0);
+ scan = index_beginscan(rel, idxrel,
+ 0, /* flags */
+ &snap, NULL, skey_attoff, 0);
retry:
found = false;
@@ -383,7 +385,7 @@ RelationFindReplTupleSeq(Relation rel, LockTupleMode lockmode,
/* Start a heap scan. */
InitDirtySnapshot(snap);
- scan = table_beginscan(rel, &snap, 0, NULL);
+ scan = table_beginscan(rel, 0, &snap, 0, NULL);
scanslot = table_slot_create(rel, NULL);
retry:
@@ -602,7 +604,7 @@ RelationFindDeletedTupleInfoSeq(Relation rel, TupleTableSlot *searchslot,
* not yet committed or those just committed prior to the scan are
* excluded in update_most_recent_deletion_info().
*/
- scan = table_beginscan(rel, SnapshotAny, 0, NULL);
+ scan = table_beginscan(rel, 0, SnapshotAny, 0, NULL);
scanslot = table_slot_create(rel, NULL);
table_rescan(scan, NULL);
@@ -666,7 +668,9 @@ RelationFindDeletedTupleInfoByIndex(Relation rel, Oid idxoid,
* not yet committed or those just committed prior to the scan are
* excluded in update_most_recent_deletion_info().
*/
- scan = index_beginscan(rel, idxrel, SnapshotAny, NULL, skey_attoff, 0);
+ scan = index_beginscan(rel, idxrel,
+ 0, /* flags */
+ SnapshotAny, NULL, skey_attoff, 0);
index_rescan(scan, skey, skey_attoff, NULL, 0);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7cf8d23c742..7e2c1b7467b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -146,6 +146,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
{
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
+ 0,
node->ss.ps.state->es_snapshot,
0,
NULL);
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index 9eab81fd1c8..5cacb4b215a 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -92,6 +92,7 @@ IndexOnlyNext(IndexOnlyScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
+ 0, /* flags */
estate->es_snapshot,
node->ioss_Instrument,
node->ioss_NumScanKeys,
@@ -791,6 +792,7 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
+ 0, /* flags */
node->ioss_Instrument,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
@@ -857,6 +859,7 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
+ 0, /* flags */
node->ioss_Instrument,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 06143e94c5a..aaef31dbbad 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -110,6 +110,7 @@ IndexNext(IndexScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->iss_RelationDesc,
+ 0, /* flags */
estate->es_snapshot,
node->iss_Instrument,
node->iss_NumScanKeys,
@@ -206,6 +207,7 @@ IndexNextWithReorder(IndexScanState *node)
*/
scandesc = index_beginscan(node->ss.ss_currentRelation,
node->iss_RelationDesc,
+ 0, /* flags */
estate->es_snapshot,
node->iss_Instrument,
node->iss_NumScanKeys,
@@ -1727,6 +1729,7 @@ ExecIndexScanInitializeDSM(IndexScanState *node,
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
+ 0, /* flags */
node->iss_Instrument,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
@@ -1791,6 +1794,7 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
+ 0, /* flags */
node->iss_Instrument,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
diff --git a/src/backend/executor/nodeSamplescan.c b/src/backend/executor/nodeSamplescan.c
index 6b0d65f752f..cf4dd6a16b4 100644
--- a/src/backend/executor/nodeSamplescan.c
+++ b/src/backend/executor/nodeSamplescan.c
@@ -294,6 +294,7 @@ tablesample_init(SampleScanState *scanstate)
{
scanstate->ss.ss_currentScanDesc =
table_beginscan_sampling(scanstate->ss.ss_currentRelation,
+ 0,
scanstate->ss.ps.state->es_snapshot,
0, NULL,
scanstate->use_bulkread,
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 8f219f60a93..376e877e87c 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -70,7 +70,7 @@ SeqNext(SeqScanState *node)
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
- estate->es_snapshot,
+ 0, estate->es_snapshot,
0, NULL);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -375,7 +375,7 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+ table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
}
/* ----------------------------------------------------------------
@@ -408,5 +408,5 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan);
+ table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 617713bde04..bacd7aa5bc4 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -243,6 +243,7 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
+ 0,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid);
@@ -460,7 +461,7 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan);
+ 0, pscan);
}
/* ----------------------------------------------------------------
@@ -494,5 +495,5 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan);
+ 0, pscan);
}
diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c
index 0ca312ac27d..919df5eef0a 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -3362,7 +3362,7 @@ check_default_partition_contents(Relation parent, Relation default_rel,
econtext = GetPerTupleExprContext(estate);
snapshot = RegisterSnapshot(GetLatestSnapshot());
tupslot = table_slot_create(part_rel, &estate->es_tupleTable);
- scan = table_beginscan(part_rel, snapshot, 0, NULL);
+ scan = table_beginscan(part_rel, 0, snapshot, 0, NULL);
/*
* Switch to per-tuple memory context and reset it for each tuple
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 53f85ccde01..0528f8166d8 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -7177,6 +7177,7 @@ get_actual_variable_endpoint(Relation heapRel,
GlobalVisTestFor(heapRel));
index_scan = index_beginscan(heapRel, indexRel,
+ 0, /* flags */
&SnapshotNonVacuumable, NULL,
1, 0);
/* Set it up for index-only scan */
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index 1a27bf060b3..24b2fda51df 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -156,6 +156,7 @@ extern void index_insert_cleanup(Relation indexRelation,
extern IndexScanDesc index_beginscan(Relation heapRelation,
Relation indexRelation,
+ uint32 flags,
Snapshot snapshot,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys);
@@ -182,6 +183,7 @@ extern void index_parallelscan_initialize(Relation heapRelation,
extern void index_parallelrescan(IndexScanDesc scan);
extern IndexScanDesc index_beginscan_parallel(Relation heaprel,
Relation indexrel,
+ uint32 flags,
IndexScanInstrumentation *instrument,
int nkeys, int norderbys,
ParallelIndexScanDesc pscan);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 9b403203006..e2e07348f37 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -95,10 +95,7 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
- /*
- * For sequential scans and bitmap heap scans. The current heap block's
- * corresponding page in the visibility map.
- */
+ /* Current heap block's corresponding page in the visibility map */
Buffer rs_vmbuffer;
/* these fields only used in page-at-a-time mode and for bitmap scans */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index ce340c076f8..80ea0b437d1 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -122,6 +122,7 @@ typedef struct ParallelBlockTableScanWorkerData *ParallelBlockTableScanWorker;
typedef struct IndexFetchTableData
{
Relation rel;
+ uint32 flags;
} IndexFetchTableData;
struct IndexScanInstrumentation;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 06084752245..e1f90f2b6a7 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -65,6 +65,16 @@ typedef enum ScanOptions
SO_TEMP_SNAPSHOT = 1 << 9,
} ScanOptions;
+/*
+ * Mask of flags that are set internally by the table_beginscan_* functions
+ * and must not be passed by callers.
+ */
+#define SO_INTERNAL_FLAGS \
+ (SO_TYPE_SEQSCAN | SO_TYPE_BITMAPSCAN | SO_TYPE_SAMPLESCAN | \
+ SO_TYPE_TIDSCAN | SO_TYPE_TIDRANGESCAN | SO_TYPE_ANALYZE | \
+ SO_ALLOW_STRAT | SO_ALLOW_SYNC | SO_ALLOW_PAGEMODE | \
+ SO_TEMP_SNAPSHOT)
+
/*
* Result codes for table_{update,delete,lock_tuple}, and for visibility
* routines inside table AMs.
@@ -420,7 +430,7 @@ typedef struct TableAmRoutine
*
* Tuples for an index scan can then be fetched via index_fetch_tuple.
*/
- struct IndexFetchTableData *(*index_fetch_begin) (Relation rel);
+ struct IndexFetchTableData *(*index_fetch_begin) (Relation rel, uint32 flags);
/*
* Reset index fetch. Typically this will release cross index fetch
@@ -871,12 +881,18 @@ extern TupleTableSlot *table_slot_create(Relation relation, List **reglist);
* A wrapper around the Table Access Method scan_begin callback, to centralize
* error checking. All calls to ->scan_begin() should go through this
* function.
+ *
+ * The caller-provided user_flags are validated against SO_INTERNAL_FLAGS to
+ * catch callers that accidentally pass scan-type or other internal flags.
*/
static TableScanDesc
table_beginscan_common(Relation rel, Snapshot snapshot, int nkeys,
ScanKeyData *key, ParallelTableScanDesc pscan,
- uint32 flags)
+ uint32 flags, uint32 user_flags)
{
+ Assert((user_flags & SO_INTERNAL_FLAGS) == 0);
+ flags |= user_flags;
+
/*
* We don't allow scans to be started while CheckXidAlive is set, except
* via systable_beginscan() et al. See detailed comments in xact.c where
@@ -893,13 +909,14 @@ table_beginscan_common(Relation rel, Snapshot snapshot, int nkeys,
* `snapshot`, and if nkeys != 0, the results are filtered by those scan keys.
*/
static inline TableScanDesc
-table_beginscan(Relation rel, Snapshot snapshot,
+table_beginscan(Relation rel, uint32 flags, Snapshot snapshot,
int nkeys, ScanKeyData *key)
{
- uint32 flags = SO_TYPE_SEQSCAN |
+ uint32 internal_flags = SO_TYPE_SEQSCAN |
SO_ALLOW_STRAT | SO_ALLOW_SYNC | SO_ALLOW_PAGEMODE;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL,
+ internal_flags, flags);
}
/*
@@ -928,7 +945,7 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
if (allow_sync)
flags |= SO_ALLOW_SYNC;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags, 0);
}
/*
@@ -938,12 +955,13 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
* make it worth using the same data structure.
*/
static inline TableScanDesc
-table_beginscan_bm(Relation rel, Snapshot snapshot,
+table_beginscan_bm(Relation rel, uint32 flags, Snapshot snapshot,
int nkeys, ScanKeyData *key)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 internal_flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL,
+ internal_flags, flags);
}
/*
@@ -954,21 +972,22 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
* also allows control of whether page-mode visibility checking is used.
*/
static inline TableScanDesc
-table_beginscan_sampling(Relation rel, Snapshot snapshot,
+table_beginscan_sampling(Relation rel, uint32 flags, Snapshot snapshot,
int nkeys, ScanKeyData *key,
bool allow_strat, bool allow_sync,
bool allow_pagemode)
{
- uint32 flags = SO_TYPE_SAMPLESCAN;
+ uint32 internal_flags = SO_TYPE_SAMPLESCAN;
if (allow_strat)
- flags |= SO_ALLOW_STRAT;
+ internal_flags |= SO_ALLOW_STRAT;
if (allow_sync)
- flags |= SO_ALLOW_SYNC;
+ internal_flags |= SO_ALLOW_SYNC;
if (allow_pagemode)
- flags |= SO_ALLOW_PAGEMODE;
+ internal_flags |= SO_ALLOW_PAGEMODE;
- return table_beginscan_common(rel, snapshot, nkeys, key, NULL, flags);
+ return table_beginscan_common(rel, snapshot, nkeys, key, NULL,
+ internal_flags, flags);
}
/*
@@ -981,7 +1000,7 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
{
uint32 flags = SO_TYPE_TIDSCAN;
- return table_beginscan_common(rel, snapshot, 0, NULL, NULL, flags);
+ return table_beginscan_common(rel, snapshot, 0, NULL, NULL, flags, 0);
}
/*
@@ -994,7 +1013,7 @@ table_beginscan_analyze(Relation rel)
{
uint32 flags = SO_TYPE_ANALYZE;
- return table_beginscan_common(rel, NULL, 0, NULL, NULL, flags);
+ return table_beginscan_common(rel, NULL, 0, NULL, NULL, flags, 0);
}
/*
@@ -1057,14 +1076,15 @@ table_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, TupleTableS
* for a TID range scan.
*/
static inline TableScanDesc
-table_beginscan_tidrange(Relation rel, Snapshot snapshot,
+table_beginscan_tidrange(Relation rel, uint32 flags, Snapshot snapshot,
ItemPointer mintid,
ItemPointer maxtid)
{
TableScanDesc sscan;
- uint32 flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
+ uint32 internal_flags = SO_TYPE_TIDRANGESCAN | SO_ALLOW_PAGEMODE;
- sscan = table_beginscan_common(rel, snapshot, 0, NULL, NULL, flags);
+ sscan = table_beginscan_common(rel, snapshot, 0, NULL, NULL,
+ internal_flags, flags);
/* Set the range of TIDs to scan */
sscan->rs_rd->rd_tableam->scan_set_tidrange(sscan, mintid, maxtid);
@@ -1139,6 +1159,7 @@ extern void table_parallelscan_initialize(Relation rel,
* Caller must hold a suitable lock on the relation.
*/
extern TableScanDesc table_beginscan_parallel(Relation relation,
+ uint32 flags,
ParallelTableScanDesc pscan);
/*
@@ -1149,6 +1170,7 @@ extern TableScanDesc table_beginscan_parallel(Relation relation,
* Caller must hold a suitable lock on the relation.
*/
extern TableScanDesc table_beginscan_parallel_tidrange(Relation relation,
+ uint32 flags,
ParallelTableScanDesc pscan);
/*
@@ -1175,8 +1197,10 @@ table_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan)
* Tuples for an index scan can then be fetched via table_index_fetch_tuple().
*/
static inline IndexFetchTableData *
-table_index_fetch_begin(Relation rel)
+table_index_fetch_begin(Relation rel, uint32 flags)
{
+ Assert((flags & SO_INTERNAL_FLAGS) == 0);
+
/*
* We don't allow scans to be started while CheckXidAlive is set, except
* via systable_beginscan() et al. See detailed comments in xact.c where
@@ -1185,7 +1209,7 @@ table_index_fetch_begin(Relation rel)
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "scan started during logical decoding");
- return rel->rd_tableam->index_fetch_begin(rel);
+ return rel->rd_tableam->index_fetch_begin(rel, flags);
}
/*
--
2.53.0
[text/x-patch] v5-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch (2.8K, ../../ab5b6aaf-90d1-4c35-b76b-6416385df6c6@vondra.me/7-v5-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch)
download | inline diff:
From ff3a51a4ef04e4fb8d74ce5c05b2927ac24b8a9c Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH v5 1/6] bufmgr: Return whether WaitReadBuffers() needed to
wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 00bc609529a..c8ec7fb4f53 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1738,12 +1738,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1805,6 +1813,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1835,6 +1844,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1851,6 +1866,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 4017896f951..a1ef04354dd 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -249,7 +249,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-30 17:28 ` Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-03-30 17:28 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
> Subject: [PATCH v5 1/6] bufmgr: Return whether WaitReadBuffers() needed to
> wait
>
> In a subsequent commit read_stream.c will use this as an input to the read
> ahead distance.
Note that this actually depends on a prior commit to work for io_uring...
> From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
> From: Tomas Vondra <tomas@vondra.me>
> Date: Thu, 19 Mar 2026 22:25:09 +0100
> Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
>
> This adds details about AIO / prefetch for executor nodes using a
> ReadStream. Right now this applies only to BitmapHeapScan, because
> that's the only scan node using a ReadStream and collecting
> instrumentation from workers.
I don't understand why that means it should be done as part of this commit,
whereas seqscans shouldn't?
> The stats are collected by the ReadStream unconditionally, i.e. without
> EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
> integer counters), it's not worth gating this by a flag.
I think this is not true anymore.
> --- a/doc/src/sgml/ref/explain.sgml
> +++ b/doc/src/sgml/ref/explain.sgml
> @@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
> TIMING [ <replaceable class="parameter">boolean</replaceable> ]
> SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
> MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
> + IO [ <replaceable class="parameter">boolean</replaceable> ]
> FORMAT { TEXT | XML | JSON | YAML }
> </synopsis>
> </refsynopsisdiv>
> @@ -298,6 +299,17 @@ ROLLBACK;
> </listitem>
> </varlistentry>
>
> + <varlistentry>
> + <term><literal>IO</literal></term>
> + <listitem>
> + <para>
> + Include information on I/O performed by each node.
> + This parameter may only be used when <literal>ANALYZE</literal> is also
> + enabled. It defaults to <literal>TRUE</literal>.
> + </para>
> + </listitem>
> + </varlistentry>
> +
> <varlistentry>
> <term><literal>FORMAT</literal></term>
> <listitem>
I wonder if it's sane to default this to true. The amount of adjustments in
the regression tests required in the later commits makes me a bit wary. But I
guess just going around and adding a few dozen OPTION OFF is what we've lately
been doing, so perhaps it's ok.
> @@ -1198,6 +1198,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> scan->rs_base.rs_nkeys = nkeys;
> scan->rs_base.rs_flags = flags;
> scan->rs_base.rs_parallel = parallel_scan;
> + scan->rs_base.rs_instrument = NULL;
> scan->rs_strategy = NULL; /* set in initscan */
> scan->rs_cbuf = InvalidBuffer;
>
> @@ -1267,6 +1268,12 @@ heap_beginscan(Relation relation, Snapshot snapshot,
>
> scan->rs_read_stream = NULL;
>
> + /* allocate instrumentation */
> + if (flags & SO_SCAN_INSTRUMENT)
> + scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
> + else
> + scan->rs_base.rs_instrument = NULL;
> +
> /*
> * Set up a read stream for sequential scans and TID range scans. This
> * should be done after initscan() because initscan() allocates the
> @@ -1296,7 +1303,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> MAIN_FORKNUM,
> cb,
> scan,
> - 0);
> + 0,
> + &scan->rs_base.rs_instrument->io);
> }
> else if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
> {
> @@ -1307,7 +1315,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> MAIN_FORKNUM,
> bitmapheap_stream_read_next,
> scan,
> - sizeof(TBMIterateResult));
> + sizeof(TBMIterateResult),
> + &scan->rs_base.rs_instrument->io);
> }
>
> scan->rs_vmbuffer = InvalidBuffer;
It doesn't seem great to dereference rs_base.rs_instrument if it's NULL, even
if that works because you then just compute the address of that. I think
you'll end up passing the offset of io in TableScanInstrumentation this way.
> +static void
> +print_io_usage(ExplainState *es, IOStats *stats)
> +{
> + /* don't print stats if there's nothing to report */
> + if (stats->prefetch_count > 0)
> + {
> + if (es->format == EXPLAIN_FORMAT_TEXT)
> + {
> + /* prefetch distance info */
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
> + (stats->distance_sum * 1.0 / stats->prefetch_count),
> + stats->distance_max,
> + stats->distance_capacity);
> + appendStringInfoChar(es->str, '\n');
> +
> + /* prefetch I/O info (only if there were actual I/Os) */
> + if (stats->io_count > 0)
> + {
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
> + stats->stall_count);
> +
> + appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
> + (stats->io_nblocks * 1.0 / stats->io_count),
> + (stats->io_in_progress * 1.0 / stats->io_count));
> +
> + appendStringInfoChar(es->str, '\n');
> + }
Why have these separate appendStringInfoChar's if you just could just include
the \n in the prior appendStringInfo?
Separately, is %.3f really the right thing? When is that degree of precision
useful here?
> diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
> index 7e2c1b7467b..d181fd7b8ff 100644
> --- a/src/backend/executor/nodeBitmapHeapscan.c
> +++ b/src/backend/executor/nodeBitmapHeapscan.c
> @@ -104,6 +104,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
> TBMIterator tbmiterator = {0};
> ParallelBitmapHeapState *pstate = node->pstate;
> dsa_area *dsa = node->ss.ps.state->es_query_dsa;
> + EState *estate = node->ss.ps.state;
>
> if (!pstate)
> {
> @@ -146,7 +147,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
> {
> node->ss.ss_currentScanDesc =
> table_beginscan_bm(node->ss.ss_currentRelation,
> - 0,
> + (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
> node->ss.ps.state->es_snapshot,
> 0,
> NULL);
> @@ -329,6 +330,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
> */
> si->exact_pages += node->stats.exact_pages;
> si->lossy_pages += node->stats.lossy_pages;
> +
> + /* collect prefetch info for this process from the read_stream */
> + ACCUMULATE_IO_STATS(&si->stats.io, &node->ss.ss_currentScanDesc->rs_instrument->io);
> }
>
> /*
I see no point in having ACCUMULATE_IO_STATS be a macro instead of a static
inline?
> +/*
> + * Update stream stats with current pinned buffer depth.
> + *
> + * Called once per buffer returned to the consumer in read_stream_next_buffer().
> + * Records the number of pinned buffers at that moment, so we can compute the
> + * average look-ahead depth.
> + */
> +static inline void
> +read_stream_update_stats_prefetch(ReadStream *stream)
> +{
> + IOStats *stats = stream->stats;
> +
> + if (stats == NULL)
> + return;
> +
> + stats->prefetch_count++;
> + stats->distance_sum += stream->pinned_buffers;
> + if (stream->pinned_buffers > stats->distance_max)
> + stats->distance_max = stream->pinned_buffers;
> +}
> +
> +/*
> + * Update stream stats about size of I/O requests.
> + *
> + * We count the number of I/O requests, size of requests (counted in blocks)
> + * and number of in-progress I/Os.
> + */
> +static inline void
> +read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
> +{
> + IOStats *stats = stream->stats;
> +
> + if (stats == NULL)
> + return;
> +
> + stats->io_count++;
> + stats->io_nblocks += nblocks;
> + stats->io_in_progress += in_progress;
> +}
> +
> +static inline void
> +read_stream_update_stats_stall(ReadStream *stream)
> +{
Wonder if we should rename "stall" to "wait", seems like it might be easier to
understand for a more general audience.
> /* ----------------------------------------------------------------
> @@ -404,9 +471,50 @@ void
> ExecSeqScanInitializeWorker(SeqScanState *node,
> ParallelWorkerContext *pwcxt)
> {
> + EState *estate = node->ss.ps.state;
> ParallelTableScanDesc pscan;
> + char *ptr;
> + Size size;
>
> pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
> node->ss.ss_currentScanDesc =
> - table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
> + table_beginscan_parallel(node->ss.ss_currentRelation,
> + (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
> + pscan);
> +
> + /*
> + * Workers don't get the pscan_len value in scan descriptor, so use the
> + * TAM callback again. The result has to match the earlier result in
> + * ExecSeqScanEstimate.
> + */
> + size = table_parallelscan_estimate(node->ss.ss_currentRelation,
> + estate->es_snapshot);
> +
That seems quite grotty...
> + ptr = (char *) pscan;
> + ptr += MAXALIGN(size);
> +
> + if (node->ss.ps.instrument)
> + node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
Seems like the appropriate instrumentation pointers should be computed by the
leader, not in the workers.
> diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
> index 97cb7206676..3fa6d838545 100644
> --- a/src/test/regress/expected/explain.out
> +++ b/src/test/regress/expected/explain.out
> @@ -93,112 +93,126 @@ select explain_filter('explain (analyze, buffers, io off, format text) select *
> Execution Time: N.N ms
> (3 rows)
>
> -select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
> - explain_filter
> ---------------------------------------------------------
> - <explain xmlns="http://www.postgresql.org/N/explain"; +
...
> + <explain xmlns="http://www.postgresql.org/N/explain"; +
...
Seems these tests should really use unaligned output, to avoid this stupid
"every line changes despite not really anything changing" type of diff.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-03-30 18:21 ` Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 17:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-03-30 18:21 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/30/26 19:28, Andres Freund wrote:
>
> Hi,
>
>> Subject: [PATCH v5 1/6] bufmgr: Return whether WaitReadBuffers() needed to
>> wait
>>
>> In a subsequent commit read_stream.c will use this as an input to the read
>> ahead distance.
>
> Note that this actually depends on a prior commit to work for io_uring...
>
Good point.
>
>> From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
>> From: Tomas Vondra <tomas@vondra.me>
>> Date: Thu, 19 Mar 2026 22:25:09 +0100
>> Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
>>
>> This adds details about AIO / prefetch for executor nodes using a
>> ReadStream. Right now this applies only to BitmapHeapScan, because
>> that's the only scan node using a ReadStream and collecting
>> instrumentation from workers.
>
> I don't understand why that means it should be done as part of this commit,
> whereas seqscans shouldn't?
>
Are you suggesting the commit adds support for all those scans (BHS,
SeqScan and TidRangeScan) or none of them? To me it seems better to have
at least some scan because of testing. But SeqScan/TidRangeScan don't
have the instrumentation infrastructure for parallel queries, and I
don't want to do that in the main patch - it seems rather unrelated. And
I also don't want to add it before the main patch.
>
>> The stats are collected by the ReadStream unconditionally, i.e. without
>> EXPLAIN ANALYZE etc. The amount of statistics is trivial (a handful of
>> integer counters), it's not worth gating this by a flag.
>
> I think this is not true anymore.
>
Yes, that's a stale comment. Will fix.
>
>> --- a/doc/src/sgml/ref/explain.sgml
>> +++ b/doc/src/sgml/ref/explain.sgml
>> @@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
>> TIMING [ <replaceable class="parameter">boolean</replaceable> ]
>> SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
>> MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
>> + IO [ <replaceable class="parameter">boolean</replaceable> ]
>> FORMAT { TEXT | XML | JSON | YAML }
>> </synopsis>
>> </refsynopsisdiv>
>> @@ -298,6 +299,17 @@ ROLLBACK;
>> </listitem>
>> </varlistentry>
>>
>> + <varlistentry>
>> + <term><literal>IO</literal></term>
>> + <listitem>
>> + <para>
>> + Include information on I/O performed by each node.
>> + This parameter may only be used when <literal>ANALYZE</literal> is also
>> + enabled. It defaults to <literal>TRUE</literal>.
>> + </para>
>> + </listitem>
>> + </varlistentry>
>> +
>> <varlistentry>
>> <term><literal>FORMAT</literal></term>
>> <listitem>
>
> I wonder if it's sane to default this to true. The amount of adjustments in
> the regression tests required in the later commits makes me a bit wary. But I
> guess just going around and adding a few dozen OPTION OFF is what we've lately
> been doing, so perhaps it's ok.
>
I'm not sure either. I initially added it as default OFF but it's true
it's similar to "buffers".
>
>
>> @@ -1198,6 +1198,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
>> scan->rs_base.rs_nkeys = nkeys;
>> scan->rs_base.rs_flags = flags;
>> scan->rs_base.rs_parallel = parallel_scan;
>> + scan->rs_base.rs_instrument = NULL;
>> scan->rs_strategy = NULL; /* set in initscan */
>> scan->rs_cbuf = InvalidBuffer;
>>
>> @@ -1267,6 +1268,12 @@ heap_beginscan(Relation relation, Snapshot snapshot,
>>
>> scan->rs_read_stream = NULL;
>>
>> + /* allocate instrumentation */
>> + if (flags & SO_SCAN_INSTRUMENT)
>> + scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
>> + else
>> + scan->rs_base.rs_instrument = NULL;
>> +
>> /*
>> * Set up a read stream for sequential scans and TID range scans. This
>> * should be done after initscan() because initscan() allocates the
>> @@ -1296,7 +1303,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
>> MAIN_FORKNUM,
>> cb,
>> scan,
>> - 0);
>> + 0,
>> + &scan->rs_base.rs_instrument->io);
>> }
>> else if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
>> {
>> @@ -1307,7 +1315,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
>> MAIN_FORKNUM,
>> bitmapheap_stream_read_next,
>> scan,
>> - sizeof(TBMIterateResult));
>> + sizeof(TBMIterateResult),
>> + &scan->rs_base.rs_instrument->io);
>> }
>>
>> scan->rs_vmbuffer = InvalidBuffer;
>
> It doesn't seem great to dereference rs_base.rs_instrument if it's NULL, even
> if that works because you then just compute the address of that. I think
> you'll end up passing the offset of io in TableScanInstrumentation this way.
>
Yes, I have already noticed / fixed this in my development branch.
>
>
>> +static void
>> +print_io_usage(ExplainState *es, IOStats *stats)
>> +{
>> + /* don't print stats if there's nothing to report */
>> + if (stats->prefetch_count > 0)
>> + {
>> + if (es->format == EXPLAIN_FORMAT_TEXT)
>> + {
>> + /* prefetch distance info */
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "Prefetch: avg=%.3f max=%d capacity=%d",
>> + (stats->distance_sum * 1.0 / stats->prefetch_count),
>> + stats->distance_max,
>> + stats->distance_capacity);
>> + appendStringInfoChar(es->str, '\n');
>> +
>> + /* prefetch I/O info (only if there were actual I/Os) */
>> + if (stats->io_count > 0)
>> + {
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "I/O: stalls=%" PRIu64,
>> + stats->stall_count);
>> +
>> + appendStringInfo(es->str, " size=%.3f inprogress=%.3f",
>> + (stats->io_nblocks * 1.0 / stats->io_count),
>> + (stats->io_in_progress * 1.0 / stats->io_count));
>> +
>> + appendStringInfoChar(es->str, '\n');
>> + }
>
> Why have these separate appendStringInfoChar's if you just could just include
> the \n in the prior appendStringInfo?
>
Not sure, IIRC it went through multiple versions and at some point the
lines were far too long (esp. with histograms etc.).
> Separately, is %.3f really the right thing? When is that degree of precision
> useful here?
>
No idea. I think %.1f would be enough. We use %.2f in a couple places,
but that's not a huge difference.
>
>> diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
>> index 7e2c1b7467b..d181fd7b8ff 100644
>> --- a/src/backend/executor/nodeBitmapHeapscan.c
>> +++ b/src/backend/executor/nodeBitmapHeapscan.c
>> @@ -104,6 +104,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
>> TBMIterator tbmiterator = {0};
>> ParallelBitmapHeapState *pstate = node->pstate;
>> dsa_area *dsa = node->ss.ps.state->es_query_dsa;
>> + EState *estate = node->ss.ps.state;
>>
>> if (!pstate)
>> {
>> @@ -146,7 +147,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
>> {
>> node->ss.ss_currentScanDesc =
>> table_beginscan_bm(node->ss.ss_currentRelation,
>> - 0,
>> + (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
>> node->ss.ps.state->es_snapshot,
>> 0,
>> NULL);
>> @@ -329,6 +330,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
>> */
>> si->exact_pages += node->stats.exact_pages;
>> si->lossy_pages += node->stats.lossy_pages;
>> +
>> + /* collect prefetch info for this process from the read_stream */
>> + ACCUMULATE_IO_STATS(&si->stats.io, &node->ss.ss_currentScanDesc->rs_instrument->io);
>> }
>>
>> /*
>
> I see no point in having ACCUMULATE_IO_STATS be a macro instead of a static
> inline?
>
Both a macro and static inline would work for me.
>
>> +/*
>> + * Update stream stats with current pinned buffer depth.
>> + *
>> + * Called once per buffer returned to the consumer in read_stream_next_buffer().
>> + * Records the number of pinned buffers at that moment, so we can compute the
>> + * average look-ahead depth.
>> + */
>> +static inline void
>> +read_stream_update_stats_prefetch(ReadStream *stream)
>> +{
>> + IOStats *stats = stream->stats;
>> +
>> + if (stats == NULL)
>> + return;
>> +
>> + stats->prefetch_count++;
>> + stats->distance_sum += stream->pinned_buffers;
>> + if (stream->pinned_buffers > stats->distance_max)
>> + stats->distance_max = stream->pinned_buffers;
>> +}
>> +
>> +/*
>> + * Update stream stats about size of I/O requests.
>> + *
>> + * We count the number of I/O requests, size of requests (counted in blocks)
>> + * and number of in-progress I/Os.
>> + */
>> +static inline void
>> +read_stream_update_stats_io(ReadStream *stream, int nblocks, int in_progress)
>> +{
>> + IOStats *stats = stream->stats;
>> +
>> + if (stats == NULL)
>> + return;
>> +
>> + stats->io_count++;
>> + stats->io_nblocks += nblocks;
>> + stats->io_in_progress += in_progress;
>> +}
>> +
>> +static inline void
>> +read_stream_update_stats_stall(ReadStream *stream)
>> +{
>
> Wonder if we should rename "stall" to "wait", seems like it might be easier to
> understand for a more general audience.
>
I agree "wait" sounds clearer.
>
>> /* ----------------------------------------------------------------
>> @@ -404,9 +471,50 @@ void
>> ExecSeqScanInitializeWorker(SeqScanState *node,
>> ParallelWorkerContext *pwcxt)
>> {
>> + EState *estate = node->ss.ps.state;
>> ParallelTableScanDesc pscan;
>> + char *ptr;
>> + Size size;
>>
>> pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
>> node->ss.ss_currentScanDesc =
>> - table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
>> + table_beginscan_parallel(node->ss.ss_currentRelation,
>> + (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
>> + pscan);
>> +
>> + /*
>> + * Workers don't get the pscan_len value in scan descriptor, so use the
>> + * TAM callback again. The result has to match the earlier result in
>> + * ExecSeqScanEstimate.
>> + */
>> + size = table_parallelscan_estimate(node->ss.ss_currentRelation,
>> + estate->es_snapshot);
>> +
>
> That seems quite grotty...
>
Yes. But what's a better solution?
>
>> + ptr = (char *) pscan;
>> + ptr += MAXALIGN(size);
>> +
>> + if (node->ss.ps.instrument)
>> + node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
>
> Seems like the appropriate instrumentation pointers should be computed by the
> leader, not in the workers.
>
This is how BitmapHeapScan does it. And how would the leader do that,
when the node is private in the worker?
>
>
>> diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
>> index 97cb7206676..3fa6d838545 100644
>> --- a/src/test/regress/expected/explain.out
>> +++ b/src/test/regress/expected/explain.out
>> @@ -93,112 +93,126 @@ select explain_filter('explain (analyze, buffers, io off, format text) select *
>> Execution Time: N.N ms
>> (3 rows)
>>
>> -select explain_filter('explain (analyze, buffers, io off, format xml) select * from int8_tbl i8');
>> - explain_filter
>> ---------------------------------------------------------
>> - <explain xmlns="http://www.postgresql.org/N/explain"; +
> ...
>> + <explain xmlns="http://www.postgresql.org/N/explain"; +
> ...
>
> Seems these tests should really use unaligned output, to avoid this stupid
> "every line changes despite not really anything changing" type of diff.
>
Probably. I can adjust that in an initial patch.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-31 13:16 ` Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-31 13:16 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Here's v6, addressing most of the review comments. Most of this is
pretty simple / non-contentious. The remaining questions are in the last
two items (phs_len field, initializing the instrumentation pointers):
- 0001 tweaks WaitReadBuffers to return if a wait was needed. As pointed
out, this does not include some changes needed for io_uring, I don't
want to make this larger. So test with "worker".
- 0002 switches a couple explain tests to unaligned output. This makes
it easier to understand output changes with yaml/json/xml output.
- I kept the three scans in separate changes. It makes it easier to
review etc. and to me it seems easier to review. I'd keep it like this
for commit, but if someone thinks we should squash it into a single
commit that's possible too.
- I kept the IO option enabled by default. I think it makes sense to
keep this aligned with BUFFERS, but on the other hand the explain should
not be overwhelming. Various other explain options are similarly
helpful, and yet we left them OFF by default.
- Cleanup of the explain formatting code (merging appendString lines),
and so on. One question is whether to keep printing the information only
when (stats->prefetch_count > 0), especially in the non-text cases.
With the three scans this is not an issue, because the read stream is
initialized right away. But with index scans we delay that a bit, so I
wonder if this might cause some test instability. Probably not.
- I switched to %.2f format for printing the averages etc. instead of
%.3f. Not a huge change, we could probably go even to %.1f ...
- Replaced the ACCUMULATE_IO_STATS macro with a static inline.
- Renamed "stalls" to "waits".
- Fixed a couple obsolete comments, and the bug in dereferencing NULL
when the instrumentation was not initialized.
- Added the pscan_len length (returned by table_parallelscan_estimate)
to ParallelTableScanDesc, so that the workers can use this to calculate
the pointer of shared instrumentation without calling the _estimate
again (which I agree seems wrong and grotty).
This only matters for SeqScan and TidRangeScan, and it makes it quite a
bit cleaner. I don't see a better way. Right now the nodes set the new
field right after calling shm_toc_allocate, but I suppose it'd be more
correct to pass it to table_parallelscan_initialize() and set it from
there. Barring objections I'll adjust this for v7.
- I still don't understand how could the leader calculate the
instrumentation pointers for workers, so it's done the same way as in
BHS (except that we need to use the new phs_len field to calculate the
initial offset, of course).
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v6-0005-show-prefetch-stats-for-TidRangeScan.patch (11.0K, ../../6f541abf-f9e1-4830-93cc-4a849dbf2ecf@vondra.me/2-v6-0005-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From f0d8fab3aa0d0abb7d8cbaa118d0a8048b2063db Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 20:55:00 +0100
Subject: [PATCH v6 5/5] show prefetch stats for TidRangeScan
- enable show_scan_io_usage for TidRangeScan
- add infrastructure to allocate/collect instrumentation from parallel
workers
---
src/backend/commands/explain.c | 32 ++++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeTidrangescan.c | 142 +++++++++++++++++++++---
src/include/executor/instrument_node.h | 11 ++
src/include/executor/nodeTidrangescan.h | 1 +
src/include/nodes/execnodes.h | 1 +
6 files changed, 177 insertions(+), 13 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ac881000788..846323606ce 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2152,6 +2152,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4111,6 +4112,27 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+ }
+ }
+
break;
}
default:
@@ -4157,6 +4179,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
stats = &instrument->stats.io;
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ TidRangeScanState *state = ((TidRangeScanState *) planstate);
+ SharedTidRangeScanInstrumentation *sinstrument = state->trss_sinstrument;
+ TidRangeScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index d1747993df0..7e1498d385c 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1128,6 +1128,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..9b6e2b91183 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -242,12 +242,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +349,34 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new state and zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -435,11 +470,23 @@ void
ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->trss_pscanlen = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
- node->trss_pscanlen =
- table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->trss_pscanlen);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -454,17 +501,50 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
+ size = MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
+ pscan = shm_toc_allocate(pcxt->toc, size);
+ pscan->phs_len = node->trss_pscanlen;
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+ }
+
+ node->trss_sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -493,12 +573,48 @@ void
ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+
+ /* set pointer to the shared instrumentation */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(pscan->phs_len);
+
+ if (node->ss.ps.instrument)
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecTidRangeScanRetrieveInstrumentation
+ *
+ * Transfer scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index dba3ad7f599..9102404d155 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -106,6 +106,17 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..4f5dd74e0fc 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -27,5 +27,6 @@ extern void ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *p
extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 90da865518c..903f8c1160c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1896,6 +1896,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
--
2.53.0
[text/x-patch] v6-0004-show-prefetch-stats-for-SeqScan.patch (40.3K, ../../6f541abf-f9e1-4830-93cc-4a849dbf2ecf@vondra.me/3-v6-0004-show-prefetch-stats-for-SeqScan.patch)
download | inline diff:
From cd8c8958b424b1c6799020787bfc238b8730a637 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:50:57 +0200
Subject: [PATCH v6 4/5] show prefetch stats for SeqScan
- enable show_scan_io_usage for SeqScan
- add infrastructure to allocate/collect instrumentation from parallel workers
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/backend/commands/explain.c | 31 ++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeSeqscan.c | 142 ++++++++++++++++--
src/include/access/relscan.h | 1 +
src/include/executor/instrument_node.h | 15 +-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 1 +
src/test/isolation/expected/merge-update.out | 2 +-
src/test/isolation/specs/merge-update.spec | 5 +-
src/test/regress/expected/explain.out | 26 +++-
src/test/regress/expected/partition_prune.out | 34 ++---
src/test/regress/sql/explain.sql | 12 +-
src/test/regress/sql/partition_prune.sql | 34 ++---
15 files changed, 255 insertions(+), 68 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 0f5271d476e..3f178eda913 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11937,7 +11937,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12183,7 +12183,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12227,7 +12227,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12387,7 +12387,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 49ed797e8ef..2aee5bb3fe9 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4059,7 +4059,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4134,13 +4134,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4192,7 +4192,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8e015c2de33..ac881000788 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4068,6 +4068,27 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
/* Initialize counters with stats from the local process first */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+ }
+ }
+
+ break;
+ }
case T_BitmapHeapScan:
{
SharedBitmapHeapInstrumentation *sinstrument
@@ -4118,6 +4139,16 @@ show_io_usage(PlanState *planstate, ExplainState *es, int worker)
/* get instrumentation for the given worker */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
+ break;
+ }
case T_BitmapHeapScan:
{
BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 755191b51ef..d1747993df0 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1125,6 +1125,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..032f3dc5efc 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -65,6 +65,14 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
@@ -72,8 +80,7 @@ SeqNext(SeqScanState *node)
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -297,6 +304,35 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
/*
* get information from node
*/
@@ -351,10 +387,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -370,17 +419,52 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ pscan = shm_toc_allocate(pcxt->toc, size);
+ pscan->phs_len = node->pscan_len;
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -409,11 +493,47 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+
+ /* set pointer to the shared instrumentation */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(pscan->phs_len);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index a9c334e633d..1c3e2f27aa7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -89,6 +89,7 @@ typedef struct ParallelTableScanDescData
bool phs_syncscan; /* report location to syncscan logic? */
bool phs_snapshot_any; /* SnapshotAny, not phs_snapshot_data? */
Size phs_snapshot_off; /* data for snapshot */
+ Size phs_len; /* length of the scan descriptor */
} ParallelTableScanDescData;
typedef struct ParallelTableScanDescData *ParallelTableScanDesc;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 6d1554fbc46..dba3ad7f599 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -89,9 +89,22 @@ AccumulateIOStats(IOStats *dst, IOStats *src)
}
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information for sequential scans
* ---------------------
*/
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
typedef struct IndexScanInstrumentation
{
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 684e398f824..90da865518c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1644,6 +1644,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..dd9c67c4dc1 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..cf387eccefc 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -33,6 +33,9 @@ setup
FOR ln IN EXECUTE $1 LOOP
-- Ignore hash memory usage because it varies depending on the system
CONTINUE WHEN (ln ~ 'Memory Usage');
+ -- Ignore prefetch and I/O because it varies depending on the system
+ CONTINUE WHEN (ln ~ 'Prefetch');
+ CONTINUE WHEN (ln ~ 'I/O');
RETURN NEXT ln;
END LOOP;
END;
@@ -165,7 +168,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d7754c6f2cb..3e504911db4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers off, io off, verbose) select *
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
@@ -286,7 +300,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
explain_filter
-------------------------------------
[ +
@@ -421,7 +435,7 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
Planning Time: N.N
(1 row)
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
explain_filter
------------------------------------
[ +
@@ -524,7 +538,7 @@ set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -749,7 +763,7 @@ select explain_filter('explain (analyze,buffers off,io off,serialize) select * f
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual rows=N.N loops=N)
@@ -758,7 +772,7 @@ select explain_filter('explain (analyze,serialize text,buffers,timing off) selec
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1012be91689..6bd15f26aec 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2315,7 +2315,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
@@ -2325,7 +2325,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
@@ -2340,7 +2340,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2376,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2687,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2864,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3019,7 +3019,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3046,7 +3046,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3097,7 +3097,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3124,7 +3124,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3194,7 +3194,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3232,7 +3232,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3478,7 +3478,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
@@ -3498,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3513,7 +3513,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3535,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3550,7 +3550,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan expr_1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index dbd210a07f2..b7907a228fa 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -65,7 +65,7 @@ explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
@@ -82,7 +82,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
set track_io_timing = off;
-- SETTINGS option
@@ -107,7 +107,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
select explain_filter('explain (memory) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
select explain_filter('explain (memory) execute int8_query');
@@ -147,7 +147,7 @@ set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -178,8 +178,8 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..c5968932aed 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,7 +669,7 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
--
2.53.0
[text/x-patch] v6-0003-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch (71.4K, ../../6f541abf-f9e1-4830-93cc-4a849dbf2ecf@vondra.me/4-v6-0003-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch)
download | inline diff:
From cd5437f5f49e84b8ae3f8cb8f0df4e605c1b5592 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:47:04 +0200
Subject: [PATCH v6 3/5] explain: show prefetch stats in EXPLAIN (ANALYZE)
This adds details about AIO / prefetch for executor nodes using a
ReadStream. Right now this applies only to BitmapHeapScan, because
that's the only scan node using a ReadStream and collecting
instrumentation from workers.
The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (enabled by default). This mimics what we do for BUFFERS,
which seems like the closest EXPLAIN option. The amount of collected
statistics is very limited, but it can be disabled if needed.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flags is specified, so
that the executor and ReadStream has access to that.
The collected stats are designed for ReadStream, but is meant to be
reasonably generic in case a TAM manages I/Os in different ways.
---
contrib/amcheck/verify_heapam.c | 3 +-
contrib/bloom/blscan.c | 3 +-
contrib/bloom/blvacuum.c | 6 +-
contrib/pg_prewarm/autoprewarm.c | 3 +-
contrib/pg_prewarm/pg_prewarm.c | 3 +-
contrib/pg_visibility/pg_visibility.c | 6 +-
contrib/pgstattuple/pgstatapprox.c | 3 +-
contrib/pgstattuple/pgstatindex.c | 6 +-
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/brin/brin.c | 3 +-
src/backend/access/gin/ginvacuum.c | 3 +-
src/backend/access/gist/gistvacuum.c | 3 +-
src/backend/access/hash/hash.c | 3 +-
src/backend/access/heap/heapam.c | 15 +-
src/backend/access/heap/vacuumlazy.c | 6 +-
src/backend/access/nbtree/nbtree.c | 3 +-
src/backend/access/spgist/spgvacuum.c | 3 +-
src/backend/commands/analyze.c | 3 +-
src/backend/commands/explain.c | 155 +++++++++++++++++-
src/backend/commands/explain_state.c | 15 ++
src/backend/executor/nodeBitmapHeapscan.c | 20 ++-
src/backend/storage/aio/read_stream.c | 91 +++++++++-
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 50 ++++++
src/include/storage/read_stream.h | 4 +-
.../modules/index/expected/killtuples.out | 44 ++---
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/modules/test_aio/test_aio.c | 3 +-
src/test/regress/expected/explain.out | 16 +-
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 2 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 16 +-
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/tools/pgindent/typedefs.list | 2 +
45 files changed, 462 insertions(+), 90 deletions(-)
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 20ff58aa782..5cfe8e24615 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -477,7 +477,8 @@ verify_heapam(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
stream_cb,
stream_data,
- 0);
+ 0,
+ NULL);
while ((ctx.buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
{
diff --git a/contrib/bloom/blscan.c b/contrib/bloom/blscan.c
index 1a0e42021ec..9bdc4a95c18 100644
--- a/contrib/bloom/blscan.c
+++ b/contrib/bloom/blscan.c
@@ -138,7 +138,8 @@ blgetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/bloom/blvacuum.c b/contrib/bloom/blvacuum.c
index 6beb1c20ebb..587757c1e18 100644
--- a/contrib/bloom/blvacuum.c
+++ b/contrib/bloom/blvacuum.c
@@ -71,7 +71,8 @@ blbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
@@ -224,7 +225,8 @@ blvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = BLOOM_HEAD_BLKNO; blkno < npages; blkno++)
{
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index ba0bc8e6d4a..4c5448f177a 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -628,7 +628,8 @@ autoprewarm_database_main(Datum main_arg)
p.forknum,
apw_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/*
* Loop until we've prewarmed all the blocks from this fork. The
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index c2716086693..e4541f46777 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -256,7 +256,8 @@ pg_prewarm(PG_FUNCTION_ARGS)
forkNumber,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (block = first_block; block <= last_block; ++block)
{
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dfab0b64cf5..f09bb68624f 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -519,7 +519,8 @@ collect_visibility_data(Oid relid, bool include_pd)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
}
for (blkno = 0; blkno < nblocks; ++blkno)
@@ -739,7 +740,8 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen)
MAIN_FORKNUM,
collect_corrupt_items_read_stream_next_block,
&p,
- 0);
+ 0,
+ NULL);
/* Loop over every block in the relation. */
while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 21e0b50fb4b..a926a1f1168 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -146,7 +146,8 @@ statapprox_heap(Relation rel, output_type *stat)
MAIN_FORKNUM,
statapprox_heap_read_stream_next,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 3a3f2637bd9..0de33558c47 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -296,7 +296,8 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
@@ -685,7 +686,8 @@ pgstathashindex(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = startblk; blkno < nblocks; blkno++)
{
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..4951b624893 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>TRUE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..652d306c7cd 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2192,7 +2192,8 @@ brin_vacuum_scan(Relation idxrel, BufferAccessStrategy strategy)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* Scan the index in physical order, and clean up any possible mess in
diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c
index 840543eb664..484360a9d6d 100644
--- a/src/backend/access/gin/ginvacuum.c
+++ b/src/backend/access/gin/ginvacuum.c
@@ -799,7 +799,8 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (blkno = GIN_ROOT_BLKNO; blkno < npages; blkno++)
{
diff --git a/src/backend/access/gist/gistvacuum.c b/src/backend/access/gist/gistvacuum.c
index 686a0418054..e4f578b2de0 100644
--- a/src/backend/access/gist/gistvacuum.c
+++ b/src/backend/access/gist/gistvacuum.c
@@ -223,7 +223,8 @@ gistvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/hash/hash.c b/src/backend/access/hash/hash.c
index 8d8cd30dc38..acd77fd6093 100644
--- a/src/backend/access/hash/hash.c
+++ b/src/backend/access/hash/hash.c
@@ -547,7 +547,8 @@ hashbulkdelete(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
hash_bulkdelete_read_stream_cb,
&stream_private,
- 0);
+ 0,
+ NULL);
bucket_loop:
while (cur_bucket <= cur_maxbucket)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d34136d2e94..b381262e644 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1199,6 +1199,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1268,6 +1269,12 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_read_stream = NULL;
+ /* allocate instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ else
+ scan->rs_base.rs_instrument = NULL;
+
/*
* Set up a read stream for sequential scans and TID range scans. This
* should be done after initscan() because initscan() allocates the
@@ -1297,7 +1304,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
cb,
scan,
- 0);
+ 0,
+ (scan->rs_base.rs_instrument) ?
+ &scan->rs_base.rs_instrument->io : NULL);
}
else if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
{
@@ -1308,7 +1317,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
MAIN_FORKNUM,
bitmapheap_stream_read_next,
scan,
- sizeof(TBMIterateResult));
+ sizeof(TBMIterateResult),
+ (scan->rs_base.rs_instrument) ?
+ &scan->rs_base.rs_instrument->io : NULL);
}
scan->rs_vmbuffer = InvalidBuffer;
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 24001b27387..8692103bd81 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -1304,7 +1304,8 @@ lazy_scan_heap(LVRelState *vacrel)
MAIN_FORKNUM,
heap_vac_scan_next_block,
vacrel,
- sizeof(bool));
+ sizeof(bool),
+ NULL);
while (true)
{
@@ -2663,7 +2664,8 @@ lazy_vacuum_heap_rel(LVRelState *vacrel)
MAIN_FORKNUM,
vacuum_reap_lp_read_stream_next,
iter,
- sizeof(TidStoreIterResult));
+ sizeof(TidStoreIterResult),
+ NULL);
while (true)
{
diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c
index 6d870e4ebe7..98b7bc5c1b8 100644
--- a/src/backend/access/nbtree/nbtree.c
+++ b/src/backend/access/nbtree/nbtree.c
@@ -1326,7 +1326,8 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult *stats,
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
for (;;)
{
/* Get the current relation length */
diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c
index c461f8dc02d..d8da3ebf43a 100644
--- a/src/backend/access/spgist/spgvacuum.c
+++ b/src/backend/access/spgist/spgvacuum.c
@@ -835,7 +835,8 @@ spgvacuumscan(spgBulkDeleteState *bds)
MAIN_FORKNUM,
block_range_read_stream_cb,
&p,
- 0);
+ 0,
+ NULL);
/*
* The outer loop iterates over all index pages except the metapage, in
diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c
index eeed91be266..a2cf176a485 100644
--- a/src/backend/commands/analyze.c
+++ b/src/backend/commands/analyze.c
@@ -1251,7 +1251,8 @@ acquire_sample_rows(Relation onerel, int elevel,
MAIN_FORKNUM,
block_sampling_read_stream_next,
&bs,
- 0);
+ 0,
+ NULL);
/* Outer loop over blocks to sample */
while (table_scan_analyze_next_block(scan, stream))
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4b70166b0e..8e015c2de33 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,8 @@
*/
#include "postgres.h"
+#include "access/genam.h"
+#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +141,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
+static void show_io_usage(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2016,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2035,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2297,8 +2306,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->walusage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
@@ -2315,6 +2324,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+ if (es->io)
+ show_io_usage(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3985,6 +3997,145 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f inprogress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+/*
+ * show_scan_io_usage
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started or no prefetch stats */
+ if (!(planstate &&
+ planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument))
+ return;
+
+ /* Initialize counters with stats from the local process first */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /*
+ * get the sum of the counters set within each and every
+ * process
+ */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
+/*
+ * show_io_usage
+ * show info about I/O prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_io_usage(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ IOStats *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -159,6 +160,11 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ io_set = true;
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -179,12 +185,21 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 73831aed451..dcd24d266eb 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -104,6 +104,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
TBMIterator tbmiterator = {0};
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
+ EState *estate = node->ss.ps.state;
if (!pstate)
{
@@ -144,13 +145,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -330,6 +338,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index cd54c1a74ac..14912fa227f 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -107,6 +108,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -172,6 +176,57 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -380,6 +435,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -544,7 +602,8 @@ read_stream_begin_impl(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
ReadStream *stream;
size_t size;
@@ -703,6 +762,11 @@ read_stream_begin_impl(int flags,
stream->seq_until_processed = InvalidBlockNumber;
stream->temporary = SmgrIsTemp(smgr);
+ /* set capacity */
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = max_pinned_buffers;
+
/*
* Skip the initial ramp-up phase if the caller says we're going to be
* reading the whole relation. This way we start out assuming we'll be
@@ -742,7 +806,8 @@ read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size)
+ size_t per_buffer_data_size,
+ IOStats *stats)
{
return read_stream_begin_impl(flags,
strategy,
@@ -752,7 +817,8 @@ read_stream_begin_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ stats);
}
/*
@@ -777,7 +843,8 @@ read_stream_begin_smgr_relation(int flags,
forknum,
callback,
callback_private_data,
- per_buffer_data_size);
+ per_buffer_data_size,
+ NULL);
}
/*
@@ -851,6 +918,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -860,6 +928,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
stream->ios_in_progress = 1;
stream->ios[0].buffer_index = oldest_buffer_index;
stream->seq_blocknum = next_blocknum + 1;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -916,12 +990,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a stall if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -981,6 +1060,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 960abf6c214..a9c334e633d 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -25,6 +25,8 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -64,6 +66,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ab2e7fc1dfe..5f822001d02 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -68,6 +68,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 2a0ff377a73..6d1554fbc46 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -40,10 +40,59 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
*/
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +120,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..b56bf925fc1 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -87,7 +88,8 @@ extern ReadStream *read_stream_begin_relation(int flags,
ForkNumber forknum,
ReadStreamBlockNumberCB callback,
void *callback_private_data,
- size_t per_buffer_data_size);
+ size_t per_buffer_data_size,
+ IOStats *stat);
extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_data);
extern BlockNumber read_stream_next_block(ReadStream *stream,
BufferAccessStrategy *strategy);
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/modules/test_aio/test_aio.c b/src/test/modules/test_aio/test_aio.c
index a8267192cb7..cac8d931683 100644
--- a/src/test/modules/test_aio/test_aio.c
+++ b/src/test/modules/test_aio/test_aio.c
@@ -897,7 +897,8 @@ read_stream_for_blocks(PG_FUNCTION_ARGS)
MAIN_FORKNUM,
read_stream_for_blocks_cb,
&stream_data,
- 0);
+ 0,
+ NULL);
for (int i = 0; i < stream_data.nblocks; i++)
{
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..d7754c6f2cb 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -68,7 +68,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +76,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -392,7 +392,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -740,7 +740,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -768,7 +768,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -778,7 +778,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -791,7 +791,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -803,7 +803,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 218972dfab8..950c0fbae69 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1012be91689 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..dbd210a07f2 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -63,8 +63,8 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
@@ -105,7 +105,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -177,17 +177,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index e39bbb65391..532df97ee90 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c9da1f91cb9..c113d634e0b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1263,6 +1263,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3101,6 +3102,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.53.0
[text/x-patch] v6-0002-switch-explain-to-unaligned-for-json-xml-yaml.patch (12.9K, ../../6f541abf-f9e1-4830-93cc-4a849dbf2ecf@vondra.me/5-v6-0002-switch-explain-to-unaligned-for-json-xml-yaml.patch)
download | inline diff:
From 1bd80f66019aea773c2b6f46212172de592efb60 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v6 2/5] switch explain to unaligned for json/xml/yaml
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.53.0
[text/x-patch] v6-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch (2.8K, ../../6f541abf-f9e1-4830-93cc-4a849dbf2ecf@vondra.me/6-v6-0001-bufmgr-Return-whether-WaitReadBuffers-needed-to-w.patch)
download | inline diff:
From c676e43b147551c900b51d82fd486a2995a1efd5 Mon Sep 17 00:00:00 2001
From: Andres Freund <andres@anarazel.de>
Date: Tue, 3 Mar 2026 16:50:50 -0500
Subject: [PATCH v6 1/5] bufmgr: Return whether WaitReadBuffers() needed to
wait
In a subsequent commit read_stream.c will use this as an input to the read
ahead distance.
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch:
---
src/backend/storage/buffer/bufmgr.c | 18 +++++++++++++++++-
src/include/storage/bufmgr.h | 2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index cd21ae3fc36..cce1b8fa2ee 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -1740,12 +1740,20 @@ ProcessReadBuffersResult(ReadBuffersOperation *operation)
Assert(operation->nblocks_done <= operation->nblocks);
}
-void
+/*
+ * Wait for the IO operation initiated by StartReadBuffers() et al to
+ * complete.
+ *
+ * Returns whether the IO operation already had completed by the time of this
+ * call.
+ */
+bool
WaitReadBuffers(ReadBuffersOperation *operation)
{
PgAioReturn *aio_ret = &operation->io_return;
IOContext io_context;
IOObject io_object;
+ bool needed_wait = false;
if (operation->persistence == RELPERSISTENCE_TEMP)
{
@@ -1810,6 +1818,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
instr_time io_start = pgstat_prepare_io_time(track_io_timing);
pgaio_wref_wait(&operation->io_wref);
+ needed_wait = true;
/*
* The IO operation itself was already counted earlier, in
@@ -1874,6 +1883,12 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CHECK_FOR_INTERRUPTS();
+ /*
+ * If the IO completed only partially, we need to perform additional
+ * work, consider that a form of having had to wait.
+ */
+ needed_wait = true;
+
/*
* This may only complete the IO partially, either because some
* buffers were already valid, or because of a partial read.
@@ -1890,6 +1905,7 @@ WaitReadBuffers(ReadBuffersOperation *operation)
CheckReadBuffersOperation(operation, true);
/* NB: READ_DONE tracepoint was already executed in completion callback */
+ return needed_wait;
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index dd41b92f944..aa61a39d9e6 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -251,7 +251,7 @@ extern bool StartReadBuffers(ReadBuffersOperation *operation,
BlockNumber blockNum,
int *nblocks,
int flags);
-extern void WaitReadBuffers(ReadBuffersOperation *operation);
+extern bool WaitReadBuffers(ReadBuffersOperation *operation);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-31 18:10 ` Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-03-31 18:10 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-03-31 15:16:57 +0200, Tomas Vondra wrote:
> From 1bd80f66019aea773c2b6f46212172de592efb60 Mon Sep 17 00:00:00 2001
> From: Tomas Vondra <tomas@vondra.me>
> Date: Tue, 31 Mar 2026 13:44:23 +0200
> Subject: [PATCH v6 2/5] switch explain to unaligned for json/xml/yaml
LGTM.
> From cd5437f5f49e84b8ae3f8cb8f0df4e605c1b5592 Mon Sep 17 00:00:00 2001
> From: Tomas Vondra <tomas@vondra.me>
> Date: Tue, 31 Mar 2026 13:47:04 +0200
> Subject: [PATCH v6 3/5] explain: show prefetch stats in EXPLAIN (ANALYZE)
>
> This adds details about AIO / prefetch for executor nodes using a
> ReadStream. Right now this applies only to BitmapHeapScan, because
> that's the only scan node using a ReadStream and collecting
> instrumentation from workers.
s/Right now/As of this commit ... support for ... will be added in subsequent commits/
Not for now, but I wonder if we ought to introduce a REPRODUCIBLE option for
EXPLAIN [ANALYZE] that sets BUFFERS, COSTS, IO, ... the right way for the
regression tests, instead having to go through and change a gazillion tests
every time.
> diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
> index 20ff58aa782..5cfe8e24615 100644
> --- a/contrib/amcheck/verify_heapam.c
> +++ b/contrib/amcheck/verify_heapam.c
> @@ -477,7 +477,8 @@ verify_heapam(PG_FUNCTION_ARGS)
> MAIN_FORKNUM,
> stream_cb,
> stream_data,
> - 0);
> + 0,
> + NULL);
>
> while ((ctx.buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
> {
Kinda wondering if we, instead of adding a NULL to a lot of places, should
instead add a
read_stream_enable_stats(stream, stats)
or such.
> --- a/src/backend/commands/explain.c
> +++ b/src/backend/commands/explain.c
> @@ -13,6 +13,8 @@
> */
> #include "postgres.h"
>
> +#include "access/genam.h"
Why genam.h?
> +#include "access/tableam.h"
I'd probably include access/relscan.h instead.
> +/*
> + * show_scan_io_usage
> + * show info about prefetching for a seq/bitmap scan
> + *
> + * Shows summary of stats for leader and workers (if any).
> + */
> +static void
> +show_scan_io_usage(ScanState *planstate, ExplainState *es)
> +{
> + Plan *plan = planstate->ps.plan;
> + IOStats stats;
> +
> + if (!es->io)
> + return;
> +
> + /* scan not started or no prefetch stats */
> + if (!(planstate &&
> + planstate->ss_currentScanDesc &&
> + planstate->ss_currentScanDesc->rs_instrument))
> + return;
> +
> + /* Initialize counters with stats from the local process first */
> + switch (nodeTag(plan))
> + {
That "local node first" comment looks a bit odd here, given that at that level
it's for a block that does both?
> + case T_BitmapHeapScan:
> + {
> + SharedBitmapHeapInstrumentation *sinstrument
> + = ((BitmapHeapScanState *) planstate)->sinstrument;
> +
> + /* collect prefetch statistics from the read stream */
> + stats = planstate->ss_currentScanDesc->rs_instrument->io;
> +
> + /*
> + * get the sum of the counters set within each and every
> + * process
> + */
> + if (sinstrument)
> + {
> + for (int i = 0; i < sinstrument->num_workers; ++i)
> + {
> + BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
> +
> + AccumulateIOStats(&stats, &winstrument->stats.io);
> + }
I'm not entirely sure how useful it is to accumulate the stats from worker
into the leader's stats. Doesn't that mean that the leader's stats can't ever
be viewed in isolation? I'm also not really clear that it's very useful to
just smush the stats of the different streams together.
> +/*
> + * show_io_usage
> + * show info about I/O prefetching for a single worker
> + *
> + * Shows prefetching stats for a parallel scan worker.
> + */
> +static void
> +show_io_usage(PlanState *planstate, ExplainState *es, int worker)
s/show_io_usage/show_worker_io_usage/?
Yes, the other functions called in the per-worker block don't have _worker_ in
them, but afaict they aren't specific to be used in the context of a worker.
> +/*
> + * Update stream stats with current pinned buffer depth.
> + *
> + * Called once per buffer returned to the consumer in read_stream_next_buffer().
> + * Records the number of pinned buffers at that moment, so we can compute the
> + * average look-ahead depth.
> + */
> +static inline void
> +read_stream_count_prefetch(ReadStream *stream)
> +{
> + IOStats *stats = stream->stats;
> +
> + if (stats == NULL)
> + return;
> +
> + stats->prefetch_count++;
> + stats->distance_sum += stream->pinned_buffers;
> + if (stream->pinned_buffers > stats->distance_max)
> + stats->distance_max = stream->pinned_buffers;
I'd probably use Max().
> @@ -380,6 +435,9 @@ read_stream_start_pending_read(ReadStream *stream)
> Assert(stream->ios_in_progress < stream->max_ios);
> stream->ios_in_progress++;
> stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
> +
> + /* update I/O stats */
> + read_stream_count_io(stream, nblocks, stream->ios_in_progress);
> }
>
> /*
> @@ -544,7 +602,8 @@ read_stream_begin_impl(int flags,
> ForkNumber forknum,
> ReadStreamBlockNumberCB callback,
> void *callback_private_data,
> - size_t per_buffer_data_size)
> + size_t per_buffer_data_size,
> + IOStats *stats)
> {
> ReadStream *stream;
> size_t size;
> @@ -703,6 +762,11 @@ read_stream_begin_impl(int flags,
> stream->seq_until_processed = InvalidBlockNumber;
> stream->temporary = SmgrIsTemp(smgr);
>
> + /* set capacity */
> + stream->stats = stats;
> + if (stream->stats)
> + stream->stats->distance_capacity = max_pinned_buffers;
> +
The comments here don't seem to add much to me.
> +/* ---------------------
> + * Instrumentation information about read streams
> + * ---------------------
> + */
> +typedef struct IOStats
> +{
> + /* number of buffers returned to consumer (for averaging distance) */
> + uint64 prefetch_count;
> +
> + /* sum of pinned_buffers sampled at each buffer return */
> + uint64 distance_sum;
> +
> + /* maximum actual pinned_buffers observed during the scan */
> + int16 distance_max;
> +
> + /* maximum possible look-ahead distance (max_pinned_buffers) */
> + int16 distance_capacity;
> +
> + /* number of waits for a read (for the I/O) */
> + uint64 wait_count;
> +
> + /* I/O stats */
> + uint64 io_count; /* number of I/Os */
> + uint64 io_nblocks; /* sum of blocks for all I/Os */
> + uint64 io_in_progress; /* sum of in-progress I/Os */
IO stats in a struct named IO stats ;)
> /* ---------------------
> * Instrumentation information for indexscans (amgettuple and amgetbitmap)
> * ---------------------
> */
> +
> typedef struct IndexScanInstrumentation
> {
> /* Index search count (incremented with pgstat_count_index_scan call) */
Spurious new newline?
> @@ -4068,6 +4068,27 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
> /* Initialize counters with stats from the local process first */
> switch (nodeTag(plan))
> {
> + case T_SeqScan:
> + {
> + SharedSeqScanInstrumentation *sinstrument
> + = ((SeqScanState *) planstate)->sinstrument;
> +
> + /* collect prefetch statistics from the read stream */
> + stats = planstate->ss_currentScanDesc->rs_instrument->io;
> +
> + /* get the sum of the counters set within each and every process */
> + if (sinstrument)
> + {
> + for (int i = 0; i < sinstrument->num_workers; ++i)
> + {
> + SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
> +
> + AccumulateIOStats(&stats, &winstrument->stats.io);
> + }
> + }
> +
> + break;
> + }
> case T_BitmapHeapScan:
> {
> SharedBitmapHeapInstrumentation *sinstrument
It's a bit sad how much repetition there is between the per-node support for
each node type. Not sure there's a great solution though.
Perhaps it'd be nicer if we instead sum up the stats in
ExecSeqScanRetrieveInstrumentation() etc?
> @@ -119,6 +119,13 @@ explain_filter
> <Actual-Rows>N.N</Actual-Rows>
> <Actual-Loops>N</Actual-Loops>
> <Disabled>false</Disabled>
> + <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
> + <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
> + <Prefetch-Capacity>N</Prefetch-Capacity>
> + <I-O-Count>N</I-O-Count>
> + <I-O-Waits>N</I-O-Waits>
> + <Average-I-O-Size>N.N</Average-I-O-Size>
> + <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
> <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
> <Shared-Read-Blocks>N</Shared-Read-Blocks>
> <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
The I-O looks a bit ridiculous. But since people here seem to like I/O much
better than IO (which I don't personally get), it's perhaps the right choice
:(. I guess we also already have that precedent due to track_io_timing.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-03-31 18:57 ` Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-31 18:57 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/31/26 20:10, Andres Freund wrote:
> Hi,
>
> On 2026-03-31 15:16:57 +0200, Tomas Vondra wrote:
>> From 1bd80f66019aea773c2b6f46212172de592efb60 Mon Sep 17 00:00:00 2001
>> From: Tomas Vondra <tomas@vondra.me>
>> Date: Tue, 31 Mar 2026 13:44:23 +0200
>> Subject: [PATCH v6 2/5] switch explain to unaligned for json/xml/yaml
>
> LGTM.
>
>
>> From cd5437f5f49e84b8ae3f8cb8f0df4e605c1b5592 Mon Sep 17 00:00:00 2001
>> From: Tomas Vondra <tomas@vondra.me>
>> Date: Tue, 31 Mar 2026 13:47:04 +0200
>> Subject: [PATCH v6 3/5] explain: show prefetch stats in EXPLAIN (ANALYZE)
>>
>> This adds details about AIO / prefetch for executor nodes using a
>> ReadStream. Right now this applies only to BitmapHeapScan, because
>> that's the only scan node using a ReadStream and collecting
>> instrumentation from workers.
>
> s/Right now/As of this commit ... support for ... will be added in subsequent commits/
>
OK
>
> Not for now, but I wonder if we ought to introduce a REPRODUCIBLE option for
> EXPLAIN [ANALYZE] that sets BUFFERS, COSTS, IO, ... the right way for the
> regression tests, instead having to go through and change a gazillion tests
> every time.
>
Yes, having an alias for options that we know are stable sounds useful.
>
>
>> diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
>> index 20ff58aa782..5cfe8e24615 100644
>> --- a/contrib/amcheck/verify_heapam.c
>> +++ b/contrib/amcheck/verify_heapam.c
>> @@ -477,7 +477,8 @@ verify_heapam(PG_FUNCTION_ARGS)
>> MAIN_FORKNUM,
>> stream_cb,
>> stream_data,
>> - 0);
>> + 0,
>> + NULL);
>>
>> while ((ctx.buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
>> {
>
> Kinda wondering if we, instead of adding a NULL to a lot of places, should
> instead add a
> read_stream_enable_stats(stream, stats)
> or such.
>
I thought about that too, and I'll probably do it that way. I didn't do
that because having too many helpers is not great, but it seems
reasonable in this case.
>
>> --- a/src/backend/commands/explain.c
>> +++ b/src/backend/commands/explain.c
>> @@ -13,6 +13,8 @@
>> */
>> #include "postgres.h"
>>
>> +#include "gaccess/genam.h"
>
> Why genam.h?
>
>
>> +#include "access/tableam.h"
>
> I'd probably include access/relscan.h instead.
>
Ooops, this is a residue of the initial approach of doing this through
the table AM callbacks. As you say, relscan.h seems enough.
>
>
>> +/*
>> + * show_scan_io_usage
>> + * show info about prefetching for a seq/bitmap scan
>> + *
>> + * Shows summary of stats for leader and workers (if any).
>> + */
>> +static void
>> +show_scan_io_usage(ScanState *planstate, ExplainState *es)
>> +{
>> + Plan *plan = planstate->ps.plan;
>> + IOStats stats;
>> +
>> + if (!es->io)
>> + return;
>> +
>> + /* scan not started or no prefetch stats */
>> + if (!(planstate &&
>> + planstate->ss_currentScanDesc &&
>> + planstate->ss_currentScanDesc->rs_instrument))
>> + return;
>> +
>> + /* Initialize counters with stats from the local process first */
>> + switch (nodeTag(plan))
>> + {
>
> That "local node first" comment looks a bit odd here, given that at that level
> it's for a block that does both?
>
Hmmm, yeah. That comment is a bit misplaced / incomplete with the
current code. Will reword.
>
>> + case T_BitmapHeapScan:
>> + {
>> + SharedBitmapHeapInstrumentation *sinstrument
>> + = ((BitmapHeapScanState *) planstate)->sinstrument;
>> +
>> + /* collect prefetch statistics from the read stream */
>> + stats = planstate->ss_currentScanDesc->rs_instrument->io;
>> +
>> + /*
>> + * get the sum of the counters set within each and every
>> + * process
>> + */
>> + if (sinstrument)
>> + {
>> + for (int i = 0; i < sinstrument->num_workers; ++i)
>> + {
>> + BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
>> +
>> + AccumulateIOStats(&stats, &winstrument->stats.io);
>> + }
>
> I'm not entirely sure how useful it is to accumulate the stats from worker
> into the leader's stats. Doesn't that mean that the leader's stats can't ever
> be viewed in isolation? I'm also not really clear that it's very useful to
> just smush the stats of the different streams together.
>
This is modeled after show_indexscan_info, which does it like that.
Maybe it's not what we should do for read streams, ofc.
You're right it means the leader stats can't be viewed in isolation, but
we also don't show the worker stats without VERBOSE, so if we didn't
accumulate the stats like this we'd not get the worker I/O stats at all.
I think merging the stats like this is needed for the same reason.
Maybe we should accumulate only without VERBOSE? Then we'd show the
complete data for EXPLAIN (ANALYZE), and per-process data (both for the
leader and workers) with EXPLAIN (ANALYZE, VERBOSE)?
>> +/* ---------------------
>> + * Instrumentation information about read streams
>> + * ---------------------
>> + */
>> +typedef struct IOStats
>> +{
>> + /* number of buffers returned to consumer (for averaging distance) */
>> + uint64 prefetch_count;
>> +
>> + /* sum of pinned_buffers sampled at each buffer return */
>> + uint64 distance_sum;
>> +
>> + /* maximum actual pinned_buffers observed during the scan */
>> + int16 distance_max;
>> +
>> + /* maximum possible look-ahead distance (max_pinned_buffers) */
>> + int16 distance_capacity;
>> +
>> + /* number of waits for a read (for the I/O) */
>> + uint64 wait_count;
>> +
>> + /* I/O stats */
>> + uint64 io_count; /* number of I/Os */
>> + uint64 io_nblocks; /* sum of blocks for all I/Os */
>> + uint64 io_in_progress; /* sum of in-progress I/Os */
>
> IO stats in a struct named IO stats ;)
>
Old MacDonald had a farm, IO IO IO ...
>
>
>> @@ -4068,6 +4068,27 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
>> /* Initialize counters with stats from the local process first */
>> switch (nodeTag(plan))
>> {
>> + case T_SeqScan:
>> + {
>> + SharedSeqScanInstrumentation *sinstrument
>> + = ((SeqScanState *) planstate)->sinstrument;
>> +
>> + /* collect prefetch statistics from the read stream */
>> + stats = planstate->ss_currentScanDesc->rs_instrument->io;
>> +
>> + /* get the sum of the counters set within each and every process */
>> + if (sinstrument)
>> + {
>> + for (int i = 0; i < sinstrument->num_workers; ++i)
>> + {
>> + SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
>> +
>> + AccumulateIOStats(&stats, &winstrument->stats.io);
>> + }
>> + }
>> +
>> + break;
>> + }
>> case T_BitmapHeapScan:
>> {
>> SharedBitmapHeapInstrumentation *sinstrument
>
> It's a bit sad how much repetition there is between the per-node support for
> each node type. Not sure there's a great solution though.
>
> Perhaps it'd be nicer if we instead sum up the stats in
> ExecSeqScanRetrieveInstrumentation() etc?
>
The duplication is not great, but wouldn't doing it in
ExecSeqScanRetrieveInstrumentation just move the duplicated code
elsewhere? I don't see how that'd be an improvement.
>> @@ -119,6 +119,13 @@ explain_filter
>> <Actual-Rows>N.N</Actual-Rows>
>> <Actual-Loops>N</Actual-Loops>
>> <Disabled>false</Disabled>
>> + <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
>> + <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
>> + <Prefetch-Capacity>N</Prefetch-Capacity>
>> + <I-O-Count>N</I-O-Count>
>> + <I-O-Waits>N</I-O-Waits>
>> + <Average-I-O-Size>N.N</Average-I-O-Size>
>> + <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
>> <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
>> <Shared-Read-Blocks>N</Shared-Read-Blocks>
>> <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
>
> The I-O looks a bit ridiculous. But since people here seem to like I/O much
> better than IO (which I don't personally get), it's perhaps the right choice
> :(. I guess we also already have that precedent due to track_io_timing.
>
Uh, I didn't realize the XML transforms it like this. I'm not all that
attached to using "I/O" (even though it's the correct spelling!), but as
you say, track_io_timing already has the same issue.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-02 15:05 ` Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-02 15:05 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Attached is v7, with a couple adjustments based on the reviews.
The preliminary patches (beginscan flags and WaitReadStream returning if
it had to wait) were already committed, so there are no dependencies for
this patch series.
- The main change is an introduction of read_stream_enable_stats(),
which installs the IOStats pointer into an existing stream. That way we
don't need a new argument for read_stream_begin_relation(), which makes
the patch quite a bit smaller. I like this.
- Reworded a couple comments, the commit message, etc.
I also noticed a new problem. The CI reported failures on FreeBSD,
showing a weird difference in the explain output. For a while I was
really confused, because it seemed like some sort of flakiness.
But it turned out to be because of debug_parallel_query=regress (which
the FreeBSD has enabled, unlike the other configs). That makes all
queries to run as if under a Gather node, but with parallel_aware=false.
But in this case the nodes may not get the instrumentation initialized,
because execParallel.c checks plan->parallel_aware before initializing
the DSM and worker.
So there's no instrumentation, and in that case explain does not show
the stats at all. This is what caused the regression failure on FreeBSD,
because the queries outputing XML/JSON/YAML use a seqscan.
Fixing this for seqscan/tidrangescan sees simple enough - we need to
call the function even with parallel_aware=false, and then do check from
the functions to terminate early if (!parallel_aware). This is how
indexscans already do that, so I modeled it after that. I left it in
separate commits 0004 and 0006, to make it more obvious.
This works, but it's a bit ugly. At least for me it makes the code much
less readable, unfortunately. I'm not sure how else to do this, though.
There's another issue, though. BitmapHeapScan has the same issue, and
needs a similar fix, but I failed to make that work. Apparently there
are some dependencies on having the DSA that I don't understand, which
means I get segfaults if I just "copy" the indexscan approach.
FWIW this is not a new issue, it affects BitmapHeapScan already.
Not the crashes, but not showing the stats sometimes. If the queries in
explain.sql used BHS instead of a seqscan, it'd fail on CI in almost the
same way.
For a while I thought it only affects debug_parallel_query=regress, in
which case maybe the best solution would be to special-case that
somehow. But that's not really true. With a parallel query, the outer
side of the plan will have parallel_aware=false, and so will be missing
the instrumentation.
Attached is a simple reproducer, forcing a parallel join with a BHS on
both sides. Notice that the explain.log (from master) shows "Heap
Blocks" only for the inner side, while the outer side has nothing.
I suppose we should fix that, but considering no one complained about it
so far (at least I'm not aware of any reports) ...
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v7-0001-switch-explain-to-unaligned-for-json-xml-yaml.patch (12.9K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/2-v7-0001-switch-explain-to-unaligned-for-json-xml-yaml.patch)
download | inline diff:
From 6e12037ace932bee1895bc37534590bcad12e956 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v7 1/6] switch explain to unaligned for json/xml/yaml
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.53.0
[text/x-patch] v7-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch (60.0K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/3-v7-0002-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch)
download | inline diff:
From e611303cc656e5b12dacf9b9e7e1c8421916dc4a Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 15:42:32 +0200
Subject: [PATCH v7 2/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
This adds details about AIO / prefetch for executor nodes using a
ReadStream. As of this commit this applies only to BitmapHeapScan,
because that's the only scan node using a ReadStream and collecting
instrumentation from workers. Support for SeqScan and TidRangeScan,
the other scan nodes using ReadStream, will be added in subsequent
commits.
The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (enabled by default). This mimics what we do for BUFFERS,
which seems like the closest EXPLAIN option. The amount of collected
statistics is very limited, but it can be disabled if needed.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flags is specified, so
that the executor and ReadStream has access to that.
The collected stats are designed for ReadStream, but is meant to be
reasonably generic in case a TAM manages I/Os in different ways.
---
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/heap/heapam.c | 9 +
src/backend/commands/explain.c | 163 +++++++++++++++++-
src/backend/commands/explain_state.c | 15 ++
src/backend/executor/nodeBitmapHeapscan.c | 20 ++-
src/backend/storage/aio/read_stream.c | 90 +++++++++-
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 50 ++++++
src/include/storage/read_stream.h | 2 +
.../modules/index/expected/killtuples.out | 44 ++---
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/regress/expected/explain.out | 16 +-
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 2 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 16 +-
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/tools/pgindent/typedefs.list | 2 +
28 files changed, 426 insertions(+), 62 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..4951b624893 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>TRUE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6bff0032db2..015943f0bf7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1199,6 +1199,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1311,6 +1312,14 @@ heap_beginscan(Relation relation, Snapshot snapshot,
sizeof(TBMIterateResult));
}
+ /* enable read stream instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ {
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ read_stream_enable_stats(scan->rs_read_stream,
+ &scan->rs_base.rs_instrument->io);
+ }
+
scan->rs_vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4b70166b0e..5e106baa088 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,7 @@
*/
#include "postgres.h"
+#include "access/relscan.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +140,11 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
+static void show_worker_io_usage(PlanState *planstate,
+ ExplainState *es,
+ int worker);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2015,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2034,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -2297,8 +2305,8 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (es->wal && planstate->instrument)
show_wal_usage(es, &planstate->instrument->walusage);
- /* Prepare per-worker buffer/WAL usage */
- if (es->workers_state && (es->buffers || es->wal) && es->verbose)
+ /* Prepare per-worker buffer/WAL/IO usage */
+ if (es->workers_state && (es->buffers || es->wal || es->io) && es->verbose)
{
WorkerInstrumentation *w = planstate->worker_instrument;
@@ -2315,6 +2323,9 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_buffer_usage(es, &instrument->bufusage);
if (es->wal)
show_wal_usage(es, &instrument->walusage);
+ if (es->io)
+ show_worker_io_usage(planstate, es, n);
+
ExplainCloseWorker(n, es);
}
}
@@ -3985,6 +3996,154 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f inprogress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+/*
+ * show_scan_io_usage
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats;
+
+ if (!es->io)
+ return;
+
+ /* scan not started or no prefetch stats */
+ if (!(planstate &&
+ planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument))
+ return;
+
+ /*
+ * Initialize counters with stats from the local process first, then
+ * accumulate data from parallel workers.
+ */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /*
+ * get the sum of the counters set within each and every
+ * process
+ */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
+/*
+ * show_io_usage
+ * show info about I/O prefetching for a single worker
+ *
+ * Shows prefetching stats for a parallel scan worker.
+ */
+static void
+show_worker_io_usage(PlanState *planstate, ExplainState *es, int worker)
+{
+ Plan *plan = planstate->plan;
+ IOStats *stats = NULL;
+
+ if (!es->io)
+ return;
+
+ /* get instrumentation for the given worker */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
+ SharedBitmapHeapInstrumentation *sinstrument = state->sinstrument;
+ BitmapHeapScanInstrumentation *instrument;
+
+ /* FIXME bitmap heap scans don't initialize instrumentation for
+ * workers in non-parallel-aware part of the plan */
+ if (!sinstrument)
+ return;
+
+ instrument = &sinstrument->sinstrument[worker];
+ stats = &instrument->stats.io;
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -159,6 +160,11 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ io_set = true;
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -179,12 +185,21 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 73831aed451..dcd24d266eb 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -104,6 +104,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
TBMIterator tbmiterator = {0};
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
+ EState *estate = node->ss.ps.state;
if (!pstate)
{
@@ -144,13 +145,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -330,6 +338,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 31f9e35dee3..fc301fd79b1 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -108,6 +109,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -173,6 +177,73 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+/*
+ * Update stream stats about waits for I/O when consuming buffers.
+ *
+ * We count the number of I/O waits while pulling buffers out of a stream.
+ */
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
+/*
+ * Enable collection of stats into the provided IOStats.
+ */
+void
+read_stream_enable_stats(ReadStream *stream, IOStats *stats)
+{
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = stream->max_pinned_buffers;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -394,6 +465,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -881,6 +955,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -896,6 +971,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
* to avoid having to effectively do another synchronous IO for
* the next block (if it were also a miss).
*/
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -952,12 +1033,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a wait if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -1031,6 +1117,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 960abf6c214..a9c334e633d 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -25,6 +25,8 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -64,6 +66,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 4647785fd35..9e3d7186868 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -69,6 +69,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 2a0ff377a73..6d1554fbc46 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -40,10 +40,59 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
*/
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +120,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..6a589510669 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -103,5 +104,6 @@ extern BlockNumber read_stream_pause(ReadStream *stream);
extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern void read_stream_enable_stats(ReadStream *stream, IOStats *stat);
#endif /* READ_STREAM_H */
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..d7754c6f2cb 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -68,7 +68,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +76,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -392,7 +392,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -740,7 +740,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -768,7 +768,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -778,7 +778,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -791,7 +791,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -803,7 +803,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 218972dfab8..950c0fbae69 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1012be91689 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..dbd210a07f2 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -63,8 +63,8 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
@@ -105,7 +105,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -177,17 +177,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index e39bbb65391..532df97ee90 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 5bc517602b1..087bb8fcd4a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1266,6 +1266,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3104,6 +3105,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.53.0
[text/x-patch] v7-0003-show-prefetch-stats-for-SeqScan.patch (40.3K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/4-v7-0003-show-prefetch-stats-for-SeqScan.patch)
download | inline diff:
From e15a544bfb83bfeeefeeb4010b8f56ce70f4598e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:50:57 +0200
Subject: [PATCH v7 3/6] show prefetch stats for SeqScan
- enable show_scan_io_usage for SeqScan
- add infrastructure to allocate/collect instrumentation from parallel workers
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/backend/commands/explain.c | 31 ++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeSeqscan.c | 142 ++++++++++++++++--
src/include/access/relscan.h | 1 +
src/include/executor/instrument_node.h | 15 +-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 1 +
src/test/isolation/expected/merge-update.out | 2 +-
src/test/isolation/specs/merge-update.spec | 5 +-
src/test/regress/expected/explain.out | 26 +++-
src/test/regress/expected/partition_prune.out | 34 ++---
src/test/regress/sql/explain.sql | 12 +-
src/test/regress/sql/partition_prune.sql | 34 ++---
15 files changed, 255 insertions(+), 68 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index ac34a1acacb..295c94934b0 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11980,7 +11980,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12226,7 +12226,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12270,7 +12270,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12430,7 +12430,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 0e218b29a29..f5f1bbe1651 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4093,7 +4093,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4168,13 +4168,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4226,7 +4226,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 5e106baa088..f91e9d0048e 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4070,6 +4070,27 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
*/
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+ }
+ }
+
+ break;
+ }
case T_BitmapHeapScan:
{
SharedBitmapHeapInstrumentation *sinstrument
@@ -4120,6 +4141,16 @@ show_worker_io_usage(PlanState *planstate, ExplainState *es, int worker)
/* get instrumentation for the given worker */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SeqScanState *state = ((SeqScanState *) planstate);
+ SharedSeqScanInstrumentation *sinstrument = state->sinstrument;
+ SeqScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
+ break;
+ }
case T_BitmapHeapScan:
{
BitmapHeapScanState *state = ((BitmapHeapScanState *) planstate);
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 755191b51ef..d1747993df0 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1125,6 +1125,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..032f3dc5efc 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -65,6 +65,14 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
@@ -72,8 +80,7 @@ SeqNext(SeqScanState *node)
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -297,6 +304,35 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
/*
* get information from node
*/
@@ -351,10 +387,23 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ node->pscan_len = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -370,17 +419,52 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ pscan = shm_toc_allocate(pcxt->toc, size);
+ pscan->phs_len = node->pscan_len;
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->pscan_len);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+ }
+
+ node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -409,11 +493,47 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+
+ /* set pointer to the shared instrumentation */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(pscan->phs_len);
+
+ if (node->ss.ps.instrument)
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index a9c334e633d..1c3e2f27aa7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -89,6 +89,7 @@ typedef struct ParallelTableScanDescData
bool phs_syncscan; /* report location to syncscan logic? */
bool phs_snapshot_any; /* SnapshotAny, not phs_snapshot_data? */
Size phs_snapshot_off; /* data for snapshot */
+ Size phs_len; /* length of the scan descriptor */
} ParallelTableScanDescData;
typedef struct ParallelTableScanDescData *ParallelTableScanDesc;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 6d1554fbc46..dba3ad7f599 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -89,9 +89,22 @@ AccumulateIOStats(IOStats *dst, IOStats *src)
}
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information for sequential scans
* ---------------------
*/
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
typedef struct IndexScanInstrumentation
{
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 090cfccf65f..e1c8a3926ac 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1666,6 +1666,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..dd9c67c4dc1 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..cf387eccefc 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -33,6 +33,9 @@ setup
FOR ln IN EXECUTE $1 LOOP
-- Ignore hash memory usage because it varies depending on the system
CONTINUE WHEN (ln ~ 'Memory Usage');
+ -- Ignore prefetch and I/O because it varies depending on the system
+ CONTINUE WHEN (ln ~ 'Prefetch');
+ CONTINUE WHEN (ln ~ 'I/O');
RETURN NEXT ln;
END LOOP;
END;
@@ -165,7 +168,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d7754c6f2cb..3e504911db4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers off, io off, verbose) select *
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
@@ -286,7 +300,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
explain_filter
-------------------------------------
[ +
@@ -421,7 +435,7 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
Planning Time: N.N
(1 row)
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
explain_filter
------------------------------------
[ +
@@ -524,7 +538,7 @@ set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -749,7 +763,7 @@ select explain_filter('explain (analyze,buffers off,io off,serialize) select * f
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual rows=N.N loops=N)
@@ -758,7 +772,7 @@ select explain_filter('explain (analyze,serialize text,buffers,timing off) selec
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1012be91689..6bd15f26aec 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2315,7 +2315,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
@@ -2325,7 +2325,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
@@ -2340,7 +2340,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2376,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2687,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2864,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3019,7 +3019,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3046,7 +3046,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3097,7 +3097,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3124,7 +3124,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3194,7 +3194,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3232,7 +3232,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3478,7 +3478,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
@@ -3498,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3513,7 +3513,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3535,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3550,7 +3550,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan expr_1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index dbd210a07f2..b7907a228fa 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -65,7 +65,7 @@ explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
@@ -82,7 +82,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
set track_io_timing = off;
-- SETTINGS option
@@ -107,7 +107,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
select explain_filter('explain (memory) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
select explain_filter('explain (memory) execute int8_query');
@@ -147,7 +147,7 @@ set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -178,8 +178,8 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..c5968932aed 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,7 +669,7 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
--
2.53.0
[text/x-patch] v7-0004-fixup-show-prefetch-stats-for-SeqScan.patch (8.9K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/5-v7-0004-fixup-show-prefetch-stats-for-SeqScan.patch)
download | inline diff:
From 0b19891ca39f986492d5079dc6406de454bc3511 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 1 Apr 2026 20:20:57 +0200
Subject: [PATCH v7 4/6] fixup: show prefetch stats for SeqScan
---
src/backend/commands/explain.c | 29 +++++----
src/backend/executor/execParallel.c | 16 ++---
src/backend/executor/nodeSeqscan.c | 94 ++++++++++++++++++++---------
3 files changed, 88 insertions(+), 51 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f91e9d0048e..20c5d720516 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4053,21 +4053,26 @@ static void
show_scan_io_usage(ScanState *planstate, ExplainState *es)
{
Plan *plan = planstate->ps.plan;
- IOStats stats;
+ IOStats stats = {0};
if (!es->io)
return;
- /* scan not started or no prefetch stats */
- if (!(planstate &&
- planstate->ss_currentScanDesc &&
- planstate->ss_currentScanDesc->rs_instrument))
- return;
-
/*
- * Initialize counters with stats from the local process first, then
- * accumulate data from parallel workers.
+ * Initialize counters with stats from the local process first.
+ *
+ * The scan descriptor may not exist, e.g. if the scan did not start,
+ * or because of debug_parallel_query=regress. We still want to collect
+ * data from workers.
*/
+ if (planstate &&
+ planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument)
+ {
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+ }
+
+ /* accumulate data from parallel workers */
switch (nodeTag(plan))
{
case T_SeqScan:
@@ -4075,9 +4080,6 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
SharedSeqScanInstrumentation *sinstrument
= ((SeqScanState *) planstate)->sinstrument;
- /* collect prefetch statistics from the read stream */
- stats = planstate->ss_currentScanDesc->rs_instrument->io;
-
/* get the sum of the counters set within each and every process */
if (sinstrument)
{
@@ -4096,9 +4098,6 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
SharedBitmapHeapInstrumentation *sinstrument
= ((BitmapHeapScanState *) planstate)->sinstrument;
- /* collect prefetch statistics from the read stream */
- stats = planstate->ss_currentScanDesc->rs_instrument->io;
-
/*
* get the sum of the counters set within each and every
* process
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index d1747993df0..c6f7ee292dd 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -250,9 +250,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
switch (nodeTag(planstate))
{
case T_SeqScanState:
- if (planstate->plan->parallel_aware)
- ExecSeqScanEstimate((SeqScanState *) planstate,
- e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanEstimate((SeqScanState *) planstate,
+ e->pcxt);
break;
case T_IndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -484,9 +484,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
switch (nodeTag(planstate))
{
case T_SeqScanState:
- if (planstate->plan->parallel_aware)
- ExecSeqScanInitializeDSM((SeqScanState *) planstate,
- d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInitializeDSM((SeqScanState *) planstate,
+ d->pcxt);
break;
case T_IndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -1366,8 +1366,8 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
switch (nodeTag(planstate))
{
case T_SeqScanState:
- if (planstate->plan->parallel_aware)
- ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 032f3dc5efc..2e4a6138eae 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -387,17 +387,25 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
Size size;
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
size = table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
+ estate->es_snapshot);
node->pscan_len = size;
/* make sure the instrumentation is properly aligned */
size = MAXALIGN(size);
/* account for instrumentation, if required */
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ if (instrument && pcxt->nworkers > 0)
{
size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
@@ -420,13 +428,20 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
SharedSeqScanInstrumentation *sinstrument = NULL;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
Size size;
- char *ptr;
uint32 flags = SO_NONE;
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
/* Recalculate the size. This needs to match ExecSeqScanEstimate. */
size = MAXALIGN(node->pscan_len);
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ if (instrument && pcxt->nworkers > 0)
{
size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
@@ -440,31 +455,38 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
- if (ScanRelIsReadOnly(&node->ss))
- flags |= SO_HINT_REL_READ_ONLY;
-
- if (estate->es_instrument)
- flags |= SO_SCAN_INSTRUMENT;
+ /* initialize the shared instrumentation (with correct alignment) */
+ if (instrument && pcxt->nworkers > 0)
+ {
+ char *ptr = (char *) pscan;
- node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+ ptr += MAXALIGN(node->pscan_len);
- /* initialize the shared instrumentation (with correct alignment) */
- ptr = (char *) pscan;
- ptr += MAXALIGN(node->pscan_len);
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
sinstrument = (SharedSeqScanInstrumentation *) ptr;
- if (sinstrument)
- {
sinstrument->num_workers = pcxt->nworkers;
/* ensure any unfilled slots will contain zeroes */
memset(sinstrument->sinstrument, 0,
pcxt->nworkers * sizeof(SeqScanInstrumentation));
+
+ node->sinstrument = sinstrument;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's shared instrumentation */
+ return;
}
- node->sinstrument = sinstrument;
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
+ node->ss.ss_currentScanDesc =
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
}
/* ----------------------------------------------------------------
@@ -493,27 +515,43 @@ void
ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
- EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
- char *ptr;
+ EState *estate = node->ss.ps.state;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
uint32 flags = SO_NONE;
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+
+ /* set pointer to the shared instrumentation */
+ if (instrument)
+ {
+ char *ptr = (char *) pscan;
+ ptr += MAXALIGN(pscan->phs_len);
+
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's SharedInfo */
+ return;
+ }
+
if (ScanRelIsReadOnly(&node->ss))
flags |= SO_HINT_REL_READ_ONLY;
if (estate->es_instrument)
flags |= SO_SCAN_INSTRUMENT;
- pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
-
- /* set pointer to the shared instrumentation */
- ptr = (char *) pscan;
- ptr += MAXALIGN(pscan->phs_len);
-
- if (node->ss.ps.instrument)
- node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
}
/* ----------------------------------------------------------------
--
2.53.0
[text/x-patch] v7-0005-show-prefetch-stats-for-TidRangeScan.patch (11.1K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/6-v7-0005-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From 2c4376dfa6e4687f0a7ac0869938665f22230c0e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 20:55:00 +0100
Subject: [PATCH v7 5/6] show prefetch stats for TidRangeScan
- enable show_scan_io_usage for TidRangeScan
- add infrastructure to allocate/collect instrumentation from parallel
workers
---
src/backend/commands/explain.c | 32 ++++++
src/backend/executor/execParallel.c | 3 +
src/backend/executor/nodeTidrangescan.c | 142 +++++++++++++++++++++---
src/include/executor/instrument_node.h | 11 ++
src/include/executor/nodeTidrangescan.h | 1 +
src/include/nodes/execnodes.h | 1 +
6 files changed, 177 insertions(+), 13 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 20c5d720516..99b64d87126 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2151,6 +2151,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4112,6 +4113,27 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ /* collect prefetch statistics from the read stream */
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+ }
+ }
+
break;
}
default:
@@ -4164,6 +4186,16 @@ show_worker_io_usage(PlanState *planstate, ExplainState *es, int worker)
instrument = &sinstrument->sinstrument[worker];
stats = &instrument->stats.io;
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ TidRangeScanState *state = ((TidRangeScanState *) planstate);
+ SharedTidRangeScanInstrumentation *sinstrument = state->trss_sinstrument;
+ TidRangeScanInstrumentation *instrument = &sinstrument->sinstrument[worker];
+
+ stats = &instrument->stats.io;
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index c6f7ee292dd..3bb5b0d1cd2 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -1128,6 +1128,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..9b6e2b91183 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -242,12 +242,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +349,34 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new state and zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -435,11 +470,23 @@ void
ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->trss_pscanlen = size;
+
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
- node->trss_pscanlen =
- table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->trss_pscanlen);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -454,17 +501,50 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ Size size;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
+ size = MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
+ pscan = shm_toc_allocate(pcxt->toc, size);
+ pscan->phs_len = node->trss_pscanlen;
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+
+ if (sinstrument)
+ {
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+ }
+
+ node->trss_sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -493,12 +573,48 @@ void
ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
+ EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ char *ptr;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+
+ /* set pointer to the shared instrumentation */
+ ptr = (char *) pscan;
+ ptr += MAXALIGN(pscan->phs_len);
+
+ if (node->ss.ps.instrument)
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+}
+
+/* ----------------------------------------------------------------
+ * ExecTidRangeScanRetrieveInstrumentation
+ *
+ * Transfer scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index dba3ad7f599..9102404d155 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -106,6 +106,17 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..4f5dd74e0fc 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -27,5 +27,6 @@ extern void ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *p
extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e1c8a3926ac..b19bc5c6005 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1918,6 +1918,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
--
2.53.0
[text/x-patch] v7-0006-fixup-show-prefetch-stats-for-TidRangeScan.patch (7.6K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/7-v7-0006-fixup-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From 1ca6d596a776c850406f338052066b4e0e6a8493 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 1 Apr 2026 20:21:36 +0200
Subject: [PATCH v7 6/6] fixup: show prefetch stats for TidRangeScan
---
src/backend/commands/explain.c | 3 -
src/backend/executor/execParallel.c | 18 ++---
src/backend/executor/nodeTidrangescan.c | 94 ++++++++++++++++++-------
3 files changed, 77 insertions(+), 38 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 99b64d87126..4491270b387 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4120,9 +4120,6 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
SharedTidRangeScanInstrumentation *sinstrument
= ((TidRangeScanState *) planstate)->trss_sinstrument;
- /* collect prefetch statistics from the read stream */
- stats = planstate->ss_currentScanDesc->rs_instrument->io;
-
/* get the sum of the counters set within each and every process */
if (sinstrument)
{
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 3bb5b0d1cd2..9edd27da2e6 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -275,9 +275,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
e->pcxt);
break;
case T_TidRangeScanState:
- if (planstate->plan->parallel_aware)
- ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
- e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
+ e->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -507,9 +507,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
d->pcxt);
break;
case T_TidRangeScanState:
- if (planstate->plan->parallel_aware)
- ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
- d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
+ d->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -1392,9 +1392,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
pwcxt);
break;
case T_TidRangeScanState:
- if (planstate->plan->parallel_aware)
- ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
- pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
+ pwcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 9b6e2b91183..7126538addc 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -471,6 +471,14 @@ ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
Size size;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
size = table_parallelscan_estimate(node->ss.ss_currentRelation,
estate->es_snapshot);
@@ -480,7 +488,7 @@ ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
size = MAXALIGN(size);
/* account for instrumentation, if required */
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ if (instrument && pcxt->nworkers > 0)
{
size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
@@ -502,15 +510,16 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
Size size;
- char *ptr;
uint32 flags = SO_NONE;
- if (ScanRelIsReadOnly(&node->ss))
- flags |= SO_HINT_REL_READ_ONLY;
-
- if (estate->es_instrument)
- flags |= SO_SCAN_INSTRUMENT;
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
size = MAXALIGN(node->trss_pscanlen);
if (node->ss.ps.instrument && pcxt->nworkers > 0)
@@ -525,26 +534,41 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
- node->ss.ss_currentScanDesc =
- table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan, flags);
/* initialize the shared instrumentation (with correct alignment) */
- ptr = (char *) pscan;
- ptr += MAXALIGN(node->trss_pscanlen);
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ if (instrument && pcxt->nworkers > 0)
+ {
+ /* initialize the shared instrumentation (with correct alignment) */
+ char *ptr = (char *) pscan;
+
+ ptr += MAXALIGN(node->trss_pscanlen);
+
sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
- if (sinstrument)
- {
sinstrument->num_workers = pcxt->nworkers;
/* ensure any unfilled slots will contain zeroes */
memset(sinstrument->sinstrument, 0,
pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+
+ node->trss_sinstrument = sinstrument;
}
- node->trss_sinstrument = sinstrument;
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's shared instrumentation */
+ return;
+ }
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
+ node->ss.ss_currentScanDesc =
+ table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -573,28 +597,46 @@ void
ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
- EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
- char *ptr;
+ EState *estate = node->ss.ps.state;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
uint32 flags = SO_NONE;
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+
+ /* set pointer to the shared instrumentation */
+ if (instrument)
+ {
+ /* set pointer to the shared instrumentation */
+ char *ptr = (char *) pscan;
+ ptr += MAXALIGN(pscan->phs_len);
+
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's SharedInfo */
+ return;
+ }
+
if (ScanRelIsReadOnly(&node->ss))
flags |= SO_HINT_REL_READ_ONLY;
if (estate->es_instrument)
flags |= SO_SCAN_INSTRUMENT;
- pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ /* XXX only start the scan for parallel-aware ones */
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
pscan, flags);
-
- /* set pointer to the shared instrumentation */
- ptr = (char *) pscan;
- ptr += MAXALIGN(pscan->phs_len);
-
- if (node->ss.ps.instrument)
- node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
}
/* ----------------------------------------------------------------
--
2.53.0
[text/x-log] explain.log (3.1K, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/8-explain.log)
download | inline:
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------
Gather (cost=2313.13..145250.05 rows=39689 width=33) (actual time=20.841..259.154 rows=62530.00 loops=1)
Output: t1.a
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=250992 read=453
-> Nested Loop (cost=2313.13..145250.05 rows=19844 width=33) (actual time=14.816..244.557 rows=31265.00 loops=2)
Output: t1.a
Buffers: shared hit=250992 read=453
Worker 0: actual time=14.771..243.071 rows=30702.00 loops=1
Buffers: shared hit=123237 read=451
Worker 1: actual time=14.860..246.043 rows=31828.00 loops=1
Buffers: shared hit=127755 read=2
-> Parallel Bitmap Heap Scan on public.t t1 (cost=2312.66..3540.40 rows=31500 width=33) (actual time=14.718..21.161 rows=31265.00 loops=2)
Output: t1.a
Recheck Cond: (t1.a < 'a'::text)
Buffers: shared hit=875 read=449
Worker 0: actual time=14.719..21.222 rows=30702.00 loops=1
Heap Blocks: exact=410
Buffers: shared hit=431 read=449
Worker 1: actual time=14.717..21.100 rows=31828.00 loops=1
Heap Blocks: exact=424
Buffers: shared hit=444
-> Bitmap Index Scan on t_a_idx (cost=0.00..2296.91 rows=62999 width=0) (actual time=14.271..14.271 rows=62530.00 loops=1)
Index Cond: (t1.a < 'a'::text)
Index Searches: 1
Buffers: shared hit=1 read=449
Worker 0: actual time=14.271..14.271 rows=62530.00 loops=1
Buffers: shared hit=1 read=449
-> Bitmap Heap Scan on public.t t2 (cost=0.47..4.49 rows=1 width=33) (actual time=0.006..0.006 rows=1.00 loops=62530)
Output: t2.a
Recheck Cond: ((t1.a = t2.a) AND (t2.a < 'a'::text))
Buffers: shared hit=250117 read=4
Worker 0: actual time=0.006..0.006 rows=1.00 loops=30702
Buffers: shared hit=122806 read=2
Worker 1: actual time=0.006..0.006 rows=1.00 loops=31828
Buffers: shared hit=127311 read=2
-> Bitmap Index Scan on t_a_idx (cost=0.00..0.47 rows=1 width=0) (actual time=0.005..0.005 rows=1.00 loops=62530)
Index Cond: ((t2.a = t1.a) AND (t2.a < 'a'::text))
Index Searches: 62530
Buffers: shared hit=187587 read=4
Worker 0: actual time=0.005..0.005 rows=1.00 loops=30702
Buffers: shared hit=92104 read=2
Worker 1: actual time=0.005..0.005 rows=1.00 loops=31828
Buffers: shared hit=95483 read=2
Planning:
Buffers: shared hit=13
Planning Time: 0.304 ms
Execution Time: 261.762 ms
(48 rows)
[application/sql] test.sql (513B, ../../3bdbc70d-ad44-494a-8aab-868b5066fe8b@vondra.me/9-test.sql)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-03 19:01 ` Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-03 19:01 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Here's v8, with a couple improvements:
1) stats for non-parallel-aware scans
It correctly shows stats for scans that are not parallel-aware.
I'd argue for BitmapHeapScan this is a live bug in 5a1e6df3b84c, so I
kept it in a separate commit 0002. It probably should be backpatched to
PG18, even if no one complained about it.
For SeqScan/TidRangeScan I've merged the changes into 0004 and 0005.
I have to admit I'm not a huge fan of the changes required to fix this.
The logic with all these instument/parallel_aware conditions seems much
less readable to me, but I don't know how else to do this. The approach
is copied from nodeIndexscan.c, which already did that in 0fbceae841cb.
2) worker stats
While working on fixing the BHS, I realized it prints the worker stats
using ExplainOpenWorker/ExplainCloseWorker, which puts-aside the worker
stats. Which is better than the original approach - it does not require
a separate function (with a lot of duplication).
The bad thing is this gives us even less control over the order of items
in the explain output.
I'm mostly happy with this, except for the decreased readability due to
the various checks of instrument/parallel_aware.
I'm also attaching SQL scripts for the three scans, with plans that
include them in places with parallel_aware=false. All plans should have
the relevant explain stats for all the scans.
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v8-0001-switch-explain-to-unaligned-for-json-xml-yaml.patch (12.9K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/2-v8-0001-switch-explain-to-unaligned-for-json-xml-yaml.patch)
download | inline diff:
From 803e7633771ae99a470f376aacf0ed7e5b047eca Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v8 1/5] switch explain to unaligned for json/xml/yaml
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.53.0
[text/x-patch] v8-0002-Show-Bitmap-Heap-Scan-stats-for-non-parallel-awar.patch (7.3K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/3-v8-0002-Show-Bitmap-Heap-Scan-stats-for-non-parallel-awar.patch)
download | inline diff:
From 6a570e087696479f1be020c13891c8c6242d91e7 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Fri, 3 Apr 2026 14:15:28 +0200
Subject: [PATCH v8 2/5] Show Bitmap Heap Scan stats for non-parallel-aware
workers
Commit 5a1e6df3b84c added information about exact/lossy pages for each
parallel worker. It however works only for parallel-aware scans, not for
other parallel scans. For example scans on the outer side of a parallel
join are not parallel-aware, and so won't report the stats. Similarly,
queries run with debug_parallel_query=regress are not parallel-aware,
which may cause regression failures due to changes in explain output.
Fixed in a way similar to 0fbceae841cb, by invoking functions always,
and only then considering parallel_aware.
Probably needs to be backpatched to 18.
---
src/backend/commands/explain.c | 2 +-
src/backend/executor/execParallel.c | 18 ++---
src/backend/executor/nodeBitmapHeapscan.c | 86 ++++++++++++++++-------
3 files changed, 71 insertions(+), 35 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4b70166b0e..8275bb2af61 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3949,7 +3949,7 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
/* Display stats for each parallel worker */
- if (planstate->pstate != NULL)
+ if (planstate->sinstrument != NULL)
{
for (int n = 0; n < planstate->sinstrument->num_workers; n++)
{
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 755191b51ef..ce377a774f8 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -290,9 +290,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
e->pcxt);
break;
case T_BitmapHeapScanState:
- if (planstate->plan->parallel_aware)
- ExecBitmapHeapEstimate((BitmapHeapScanState *) planstate,
- e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapEstimate((BitmapHeapScanState *) planstate,
+ e->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -522,9 +522,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
d->pcxt);
break;
case T_BitmapHeapScanState:
- if (planstate->plan->parallel_aware)
- ExecBitmapHeapInitializeDSM((BitmapHeapScanState *) planstate,
- d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInitializeDSM((BitmapHeapScanState *) planstate,
+ d->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -1400,9 +1400,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
pwcxt);
break;
case T_BitmapHeapScanState:
- if (planstate->plan->parallel_aware)
- ExecBitmapHeapInitializeWorker((BitmapHeapScanState *) planstate,
- pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInitializeWorker((BitmapHeapScanState *) planstate,
+ pwcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 73831aed451..0c580365e9c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -495,9 +495,21 @@ void
ExecBitmapHeapEstimate(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
- Size size;
+ Size size = 0;
+ bool instrument = (node->ss.ps.instrument != NULL);
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ /* parallel state is needed only in parallel-aware scans */
+ if (parallel_aware)
+ {
+ size = MAXALIGN(sizeof(ParallelBitmapHeapState));
+ }
/* account for instrumentation, if required */
if (node->ss.ps.instrument && pcxt->nworkers > 0)
@@ -521,48 +533,61 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
ParallelBitmapHeapState *pstate;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
SharedBitmapHeapInstrumentation *sinstrument = NULL;
- dsa_area *dsa = node->ss.ps.state->es_query_dsa;
char *ptr;
- Size size;
+ Size size = 0;
- /* If there's no DSA, there are no workers; initialize nothing. */
- if (dsa == NULL)
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
return;
+ }
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ /* parallel state is needed only in parallel-aware scans */
+ if (parallel_aware)
+ {
+ size = MAXALIGN(sizeof(ParallelBitmapHeapState));
+ }
+
+ if (instrument && pcxt->nworkers > 0)
{
size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
}
ptr = shm_toc_allocate(pcxt->toc, size);
- pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, ptr);
+
+ if (parallel_aware)
+ {
+ pstate = (ParallelBitmapHeapState *) ptr;
+ ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
- pstate->tbmiterator = 0;
+ pstate->tbmiterator = 0;
- /* Initialize the mutex */
- SpinLockInit(&pstate->mutex);
- pstate->state = BM_INITIAL;
+ /* Initialize the mutex */
+ SpinLockInit(&pstate->mutex);
+ pstate->state = BM_INITIAL;
- ConditionVariableInit(&pstate->cv);
+ ConditionVariableInit(&pstate->cv);
- if (sinstrument)
+ node->pstate = pstate;
+ }
+
+ if (instrument && pcxt->nworkers > 0)
{
+ sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+
sinstrument->num_workers = pcxt->nworkers;
/* ensure any unfilled slots will contain zeroes */
memset(sinstrument->sinstrument, 0,
pcxt->nworkers * sizeof(BitmapHeapScanInstrumentation));
- }
- shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pstate);
- node->pstate = pstate;
- node->sinstrument = sinstrument;
+ node->sinstrument = sinstrument;
+ }
}
/* ----------------------------------------------------------------
@@ -600,16 +625,27 @@ void
ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
char *ptr;
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
Assert(node->ss.ps.state->es_query_dsa != NULL);
ptr = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- node->pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
+ if (parallel_aware)
+ {
+ node->pstate = (ParallelBitmapHeapState *) ptr;
+ ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
+ }
- if (node->ss.ps.instrument)
+ if (instrument)
node->sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
}
--
2.53.0
[text/x-patch] v8-0003-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch (58.5K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/4-v8-0003-explain-show-prefetch-stats-in-EXPLAIN-ANALYZE.patch)
download | inline diff:
From 9a2be26eb578afeb4037a9a77d620bc1b3e4a46f Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 15:42:32 +0200
Subject: [PATCH v8 3/5] explain: show prefetch stats in EXPLAIN (ANALYZE)
This adds details about AIO / prefetch for executor nodes using a
ReadStream. As of this commit this applies only to BitmapHeapScan,
because that's the only scan node using a ReadStream and collecting
instrumentation from workers. Support for SeqScan and TidRangeScan,
the other scan nodes using ReadStream, will be added in subsequent
commits.
The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (enabled by default). This mimics what we do for BUFFERS,
which seems like the closest EXPLAIN option. The amount of collected
statistics is very limited, but it can be disabled if needed.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flags is specified, so
that the executor and ReadStream has access to that.
The collected stats are designed for ReadStream, but is meant to be
reasonably generic in case a TAM manages I/Os in different ways.
---
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/heap/heapam.c | 9 ++
src/backend/commands/explain.c | 123 ++++++++++++++++++
src/backend/commands/explain_state.c | 15 +++
src/backend/executor/nodeBitmapHeapscan.c | 20 ++-
src/backend/storage/aio/read_stream.c | 90 ++++++++++++-
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 50 +++++++
src/include/storage/read_stream.h | 2 +
.../modules/index/expected/killtuples.out | 44 +++----
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/regress/expected/explain.out | 16 +--
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 2 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 16 +--
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/tools/pgindent/typedefs.list | 2 +
28 files changed, 388 insertions(+), 60 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..4951b624893 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>TRUE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6bff0032db2..015943f0bf7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1199,6 +1199,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1311,6 +1312,14 @@ heap_beginscan(Relation relation, Snapshot snapshot,
sizeof(TBMIterateResult));
}
+ /* enable read stream instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ {
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ read_stream_enable_stats(scan->rs_read_stream,
+ &scan->rs_base.rs_instrument->io);
+ }
+
scan->rs_vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8275bb2af61..fdcb7f30fe9 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,7 @@
*/
#include "postgres.h"
+#include "access/relscan.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +140,8 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2012,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -2027,6 +2031,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -3985,6 +3990,124 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f inprogress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+/*
+ * show_scan_io_usage
+ * show info about prefetching for a seq/bitmap scan
+ *
+ * Shows summary of stats for leader and workers (if any).
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats;
+
+ if (!es->io)
+ return;
+
+ /*
+ * Initialize counters with stats from the local process first.
+ *
+ * The scan descriptor may not exist, e.g. if the scan did not start,
+ * or because of debug_parallel_query=regress. We still want to collect
+ * data from workers.
+ */
+ if (planstate &&
+ planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument)
+ {
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+ }
+
+ /*
+ * Initialize counters with stats from the local process first, then
+ * accumulate data from parallel workers.
+ */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ /*
+ * get the sum of the counters set within each and every
+ * process
+ */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -159,6 +160,11 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ io_set = true;
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -179,12 +185,21 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 0c580365e9c..c13cf8679a6 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -104,6 +104,7 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
TBMIterator tbmiterator = {0};
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
+ EState *estate = node->ss.ps.state;
if (!pstate)
{
@@ -144,13 +145,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -330,6 +338,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 31f9e35dee3..fc301fd79b1 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -72,6 +72,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -108,6 +109,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -173,6 +177,73 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+/*
+ * Update stream stats about waits for I/O when consuming buffers.
+ *
+ * We count the number of I/O waits while pulling buffers out of a stream.
+ */
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
+/*
+ * Enable collection of stats into the provided IOStats.
+ */
+void
+read_stream_enable_stats(ReadStream *stream, IOStats *stats)
+{
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = stream->max_pinned_buffers;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -394,6 +465,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -881,6 +955,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -896,6 +971,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
* to avoid having to effectively do another synchronous IO for
* the next block (if it were also a miss).
*/
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -952,12 +1033,17 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
{
int16 io_index = stream->oldest_io_index;
int32 distance; /* wider temporary value, clamped below */
+ bool needed_wait;
/* Sanity check that we still agree on the buffers. */
Assert(stream->ios[io_index].op.buffers ==
&stream->buffers[oldest_buffer_index]);
- WaitReadBuffers(&stream->ios[io_index].op);
+ needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+
+ /* Count it as a wait if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
@@ -1031,6 +1117,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 960abf6c214..a9c334e633d 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -25,6 +25,8 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -64,6 +66,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 4647785fd35..9e3d7186868 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -69,6 +69,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 2a0ff377a73..6d1554fbc46 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -40,10 +40,59 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
*/
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
@@ -71,6 +120,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..6a589510669 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -103,5 +104,6 @@ extern BlockNumber read_stream_pause(ReadStream *stream);
extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern void read_stream_enable_stats(ReadStream *stream, IOStats *stat);
#endif /* READ_STREAM_H */
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..d7754c6f2cb 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -68,7 +68,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +76,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -392,7 +392,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -740,7 +740,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -768,7 +768,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -778,7 +778,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -791,7 +791,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -803,7 +803,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 218972dfab8..950c0fbae69 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1012be91689 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..dbd210a07f2 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -63,8 +63,8 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
@@ -105,7 +105,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -177,17 +177,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index e39bbb65391..532df97ee90 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 91b1225da82..cad2b71b32b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1268,6 +1268,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3107,6 +3108,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.53.0
[text/x-patch] v8-0004-show-prefetch-stats-for-SeqScan.patch (42.4K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/5-v8-0004-show-prefetch-stats-for-SeqScan.patch)
download | inline diff:
From d89ae2928dad0b9081f6aa1cf839b59a8f374e5e Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:50:57 +0200
Subject: [PATCH v8 4/5] show prefetch stats for SeqScan
- enable show_scan_io_usage for SeqScan
- add infrastructure to allocate/collect instrumentation from parallel workers
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/backend/commands/explain.c | 32 ++-
src/backend/executor/execParallel.c | 19 +-
src/backend/executor/nodeSeqscan.c | 182 ++++++++++++++++--
src/include/access/relscan.h | 1 +
src/include/executor/instrument_node.h | 15 +-
src/include/executor/nodeSeqscan.h | 1 +
src/include/nodes/execnodes.h | 1 +
src/test/isolation/expected/merge-update.out | 2 +-
src/test/isolation/specs/merge-update.spec | 5 +-
src/test/regress/expected/explain.out | 26 ++-
src/test/regress/expected/partition_prune.out | 34 ++--
src/test/regress/sql/explain.sql | 12 +-
src/test/regress/sql/partition_prune.sql | 34 ++--
15 files changed, 298 insertions(+), 82 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index ac34a1acacb..295c94934b0 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11980,7 +11980,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12226,7 +12226,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12270,7 +12270,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12430,7 +12430,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 0e218b29a29..f5f1bbe1651 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4093,7 +4093,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4168,13 +4168,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4226,7 +4226,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index fdcb7f30fe9..38403576d42 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4047,7 +4047,7 @@ static void
show_scan_io_usage(ScanState *planstate, ExplainState *es)
{
Plan *plan = planstate->ps.plan;
- IOStats stats;
+ IOStats stats = {0};
if (!es->io)
return;
@@ -4066,12 +4066,34 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
stats = planstate->ss_currentScanDesc->rs_instrument->io;
}
- /*
- * Initialize counters with stats from the local process first, then
- * accumulate data from parallel workers.
- */
+ /* accumulate data from parallel workers */
switch (nodeTag(plan))
{
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
+ break;
+ }
case T_BitmapHeapScan:
{
SharedBitmapHeapInstrumentation *sinstrument
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index ce377a774f8..9862c922afa 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -250,9 +250,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
switch (nodeTag(planstate))
{
case T_SeqScanState:
- if (planstate->plan->parallel_aware)
- ExecSeqScanEstimate((SeqScanState *) planstate,
- e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanEstimate((SeqScanState *) planstate,
+ e->pcxt);
break;
case T_IndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -484,9 +484,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
switch (nodeTag(planstate))
{
case T_SeqScanState:
- if (planstate->plan->parallel_aware)
- ExecSeqScanInitializeDSM((SeqScanState *) planstate,
- d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInitializeDSM((SeqScanState *) planstate,
+ d->pcxt);
break;
case T_IndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -1125,6 +1125,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
@@ -1363,8 +1366,8 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
switch (nodeTag(planstate))
{
case T_SeqScanState:
- if (planstate->plan->parallel_aware)
- ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..bdc91a279e0 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -65,6 +65,14 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
@@ -72,8 +80,7 @@ SeqNext(SeqScanState *node)
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -297,6 +304,35 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new SeqScanState and
+ * zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
/*
* get information from node
*/
@@ -351,10 +387,31 @@ ExecSeqScanEstimate(SeqScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+ Size size;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->pscan_len = size;
- node->pscan_len = table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -370,17 +427,66 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedSeqScanInstrumentation *sinstrument = NULL;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+ Size size;
+ uint32 flags = SO_NONE;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ /* Recalculate the size. This needs to match ExecSeqScanEstimate. */
+ size = MAXALIGN(node->pscan_len);
+ if (instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedSeqScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ }
+
+ pscan = shm_toc_allocate(pcxt->toc, size);
+ pscan->phs_offset_ins = MAXALIGN(node->pscan_len);
- pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
+ /* initialize the shared instrumentation (with correct alignment) */
+ if (instrument && pcxt->nworkers > 0)
+ {
+ char *ptr = (char *) pscan;
+
+ ptr += MAXALIGN(node->pscan_len);
+
+ sinstrument = (SharedSeqScanInstrumentation *) ptr;
+
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(SeqScanInstrumentation));
+
+ node->sinstrument = sinstrument;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's shared instrumentation */
+ return;
+ }
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
}
/* ----------------------------------------------------------------
@@ -410,10 +516,62 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ EState *estate = node->ss.ps.state;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+ uint32 flags = SO_NONE;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+
+ /* set pointer to the shared instrumentation */
+ if (instrument)
+ {
+ char *ptr = (char *) pscan;
+ ptr += pscan->phs_offset_ins;
+
+ node->sinstrument = (SharedSeqScanInstrumentation *) ptr;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's SharedInfo */
+ return;
+ }
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+}
+
+/* ----------------------------------------------------------------
+ * ExecSeqScanRetrieveInstrumentation
+ *
+ * Transfer seq scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index a9c334e633d..46cf957ee0a 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -89,6 +89,7 @@ typedef struct ParallelTableScanDescData
bool phs_syncscan; /* report location to syncscan logic? */
bool phs_snapshot_any; /* SnapshotAny, not phs_snapshot_data? */
Size phs_snapshot_off; /* data for snapshot */
+ Size phs_offset_ins; /* length of the scan descriptor */
} ParallelTableScanDescData;
typedef struct ParallelTableScanDescData *ParallelTableScanDesc;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 6d1554fbc46..dba3ad7f599 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -89,9 +89,22 @@ AccumulateIOStats(IOStats *dst, IOStats *src)
}
/* ---------------------
- * Instrumentation information for indexscans (amgettuple and amgetbitmap)
+ * Instrumentation information for sequential scans
* ---------------------
*/
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
typedef struct IndexScanInstrumentation
{
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..e2122bfffe3 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -27,5 +27,6 @@ extern void ExecSeqScanInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt);
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 090cfccf65f..fd0642608a8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1666,6 +1666,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ SharedSeqScanInstrumentation *sinstrument; /* statistics for workers */
} SeqScanState;
/* ----------------
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..dd9c67c4dc1 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..cf387eccefc 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -33,6 +33,9 @@ setup
FOR ln IN EXECUTE $1 LOOP
-- Ignore hash memory usage because it varies depending on the system
CONTINUE WHEN (ln ~ 'Memory Usage');
+ -- Ignore prefetch and I/O because it varies depending on the system
+ CONTINUE WHEN (ln ~ 'Prefetch');
+ CONTINUE WHEN (ln ~ 'I/O');
RETURN NEXT ln;
END LOOP;
END;
@@ -165,7 +168,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d7754c6f2cb..3e504911db4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers off, io off, verbose) select *
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
@@ -286,7 +300,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
explain_filter
-------------------------------------
[ +
@@ -421,7 +435,7 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
Planning Time: N.N
(1 row)
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
explain_filter
------------------------------------
[ +
@@ -524,7 +538,7 @@ set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -749,7 +763,7 @@ select explain_filter('explain (analyze,buffers off,io off,serialize) select * f
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual rows=N.N loops=N)
@@ -758,7 +772,7 @@ select explain_filter('explain (analyze,serialize text,buffers,timing off) selec
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1012be91689..6bd15f26aec 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2315,7 +2315,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
@@ -2325,7 +2325,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
@@ -2340,7 +2340,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2376,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2687,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2864,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3019,7 +3019,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3046,7 +3046,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3097,7 +3097,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3124,7 +3124,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3194,7 +3194,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3232,7 +3232,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3478,7 +3478,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
@@ -3498,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3513,7 +3513,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3535,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3550,7 +3550,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan expr_1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index dbd210a07f2..b7907a228fa 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -65,7 +65,7 @@ explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
@@ -82,7 +82,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
set track_io_timing = off;
-- SETTINGS option
@@ -107,7 +107,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
select explain_filter('explain (memory) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
select explain_filter('explain (memory) execute int8_query');
@@ -147,7 +147,7 @@ set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -178,8 +178,8 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..c5968932aed 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,7 +669,7 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
--
2.53.0
[text/x-patch] v8-0005-show-prefetch-stats-for-TidRangeScan.patch (13.0K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/6-v8-0005-show-prefetch-stats-for-TidRangeScan.patch)
download | inline diff:
From 47429d21a6e29d82a101f3cc8859305ae215511d Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Wed, 18 Mar 2026 20:55:00 +0100
Subject: [PATCH v8 5/5] show prefetch stats for TidRangeScan
- enable show_scan_io_usage for TidRangeScan
- add infrastructure to allocate/collect instrumentation from parallel
workers
---
src/backend/commands/explain.c | 26 ++++
src/backend/executor/execParallel.c | 21 +--
src/backend/executor/nodeTidrangescan.c | 184 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 11 ++
src/include/executor/nodeTidrangescan.h | 1 +
src/include/nodes/execnodes.h | 1 +
6 files changed, 222 insertions(+), 22 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 38403576d42..03408d6f42d 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2148,6 +2148,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4120,6 +4121,31 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ /* get the sum of the counters set within each and every process */
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9862c922afa..e96eca445be 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -275,9 +275,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
e->pcxt);
break;
case T_TidRangeScanState:
- if (planstate->plan->parallel_aware)
- ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
- e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
+ e->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -507,9 +507,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
d->pcxt);
break;
case T_TidRangeScanState:
- if (planstate->plan->parallel_aware)
- ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
- d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
+ d->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -1128,6 +1128,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
@@ -1389,9 +1392,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
pwcxt);
break;
case T_TidRangeScanState:
- if (planstate->plan->parallel_aware)
- ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
- pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
+ pwcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..b696648ef40 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -242,12 +242,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +349,34 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /*
+ * When ending a parallel worker, copy the statistics gathered by the
+ * worker back into shared memory so that it can be picked up by the main
+ * process to report in EXPLAIN ANALYZE.
+ */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ /*
+ * Here we accumulate the stats rather than performing memcpy on
+ * node->stats into si. When a Gather/GatherMerge node finishes it
+ * will perform planner shutdown on the workers. On rescan it will
+ * spin up new workers which will have a new state and zeroed stats.
+ */
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -435,11 +470,31 @@ void
ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
+ Size size;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ size = table_parallelscan_estimate(node->ss.ss_currentRelation,
+ estate->es_snapshot);
+ node->trss_pscanlen = size;
- node->trss_pscanlen =
- table_parallelscan_estimate(node->ss.ss_currentRelation,
- estate->es_snapshot);
- shm_toc_estimate_chunk(&pcxt->estimator, node->trss_pscanlen);
+ /* make sure the instrumentation is properly aligned */
+ size = MAXALIGN(size);
+
+ /* account for instrumentation, if required */
+ if (instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -454,17 +509,66 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ SharedTidRangeScanInstrumentation *sinstrument = NULL;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+ Size size;
+ uint32 flags = SO_NONE;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
+
+ size = MAXALIGN(node->trss_pscanlen);
+ if (node->ss.ps.instrument && pcxt->nworkers > 0)
+ {
+ size = add_size(size, offsetof(SharedTidRangeScanInstrumentation, sinstrument));
+ size = add_size(size, mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ }
- pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
+ pscan = shm_toc_allocate(pcxt->toc, size);
+ pscan->phs_offset_ins = MAXALIGN(node->trss_pscanlen);
table_parallelscan_initialize(node->ss.ss_currentRelation,
pscan,
estate->es_snapshot);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
+
+ /* initialize the shared instrumentation (with correct alignment) */
+ if (instrument && pcxt->nworkers > 0)
+ {
+ /* initialize the shared instrumentation (with correct alignment) */
+ char *ptr = (char *) pscan;
+
+ ptr += MAXALIGN(node->trss_pscanlen);
+
+ sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+
+ sinstrument->num_workers = pcxt->nworkers;
+
+ /* ensure any unfilled slots will contain zeroes */
+ memset(sinstrument->sinstrument, 0,
+ pcxt->nworkers * sizeof(TidRangeScanInstrumentation));
+
+ node->trss_sinstrument = sinstrument;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's shared instrumentation */
+ return;
+ }
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -494,11 +598,65 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ EState *estate = node->ss.ps.state;
+ bool instrument = node->ss.ps.instrument != NULL;
+ bool parallel_aware = node->ss.ps.plan->parallel_aware;
+ uint32 flags = SO_NONE;
+
+ if (!instrument && !parallel_aware)
+ {
+ /* No DSM required by the scan */
+ return;
+ }
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+
+ /* set pointer to the shared instrumentation */
+ if (instrument)
+ {
+ /* set pointer to the shared instrumentation */
+ char *ptr = (char *) pscan;
+ ptr += pscan->phs_offset_ins;
+
+ node->trss_sinstrument = (SharedTidRangeScanInstrumentation *) ptr;
+ }
+
+ if (!parallel_aware)
+ {
+ /* Only here to set up worker node's SharedInfo */
+ return;
+ }
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
+ /* XXX only start the scan for parallel-aware ones */
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+}
+
+/* ----------------------------------------------------------------
+ * ExecTidRangeScanRetrieveInstrumentation
+ *
+ * Transfer scan statistics from DSM to private memory.
+ * ----------------------------------------------------------------
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index dba3ad7f599..9102404d155 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -106,6 +106,17 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
typedef struct IndexScanInstrumentation
{
/* Index search count (incremented with pgstat_count_index_scan call) */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..4f5dd74e0fc 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -27,5 +27,6 @@ extern void ExecTidRangeScanEstimate(TidRangeScanState *node, ParallelContext *p
extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index fd0642608a8..d3c77af4589 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1918,6 +1918,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
--
2.53.0
[application/sql] test-bitmapscan.sql (1.7K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/7-test-bitmapscan.sql)
download
[application/sql] test-seqscan.sql (1.8K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/8-test-seqscan.sql)
download
[application/sql] test-tidrangescan.sql (1.8K, ../../fce326bb-1210-4d48-8c97-bb3bca396eba@vondra.me/9-test-tidrangescan.sql)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-05 22:18 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-05 22:18 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, Apr 3, 2026 at 3:01 PM Tomas Vondra <tomas@vondra.me> wrote:
>
> 1) stats for non-parallel-aware scans
>
> It correctly shows stats for scans that are not parallel-aware.
<--snip-->
> For SeqScan/TidRangeScan I've merged the changes into 0004 and 0005.
>
> I have to admit I'm not a huge fan of the changes required to fix this.
> The logic with all these instument/parallel_aware conditions seems much
> less readable to me, but I don't know how else to do this. The approach
> is copied from nodeIndexscan.c, which already did that in 0fbceae841cb.
While this is the opposite direction of what I suggested to fix BHS in
[1], what if we allocated the instrumentation and parallel-aware state
separately and accessed them with their own keys? It's a little janky
because what key could we use besides the plan_node_id, but if we add
a key-sized offset to the plan node id, we can functionally have two
separate keys.
We could do this for all nodes that have both shared instrumentation
and parallel-aware shared state. It introduces some boilerplate
functions, but I do find it easier to understand.
I've attached patches that implement this to make the idea more clear.
0002 does the two allocations for the existing index[-only] scan
nodes. 0003 uses this method to fix BHS. 0004 is your patch to add
prefetch stuff to explain and show it for BHS. And 0005 and 0006 are
using the two allocation approach for seqscan and tidrange scan. There
were some test changes you had that didn't seem required to make tests
pass in the commits that I stuffed into 0007. I used an LLM to do some
of the boilerplate and rebasing, so there might be some weirdness in
there.
If we don't do the above, then I think your current approach is the
only other realistic option. We can't do what I suggested for BHS in
[1] and always allocate the parallel-aware state because that state is
much larger for sequential scans and TID range scans.
- Melanie
[1] https://www.postgresql.org/message-id/CAAKRu_a_c8HAtJ8Ynz-dU%3DJb2PzheW0zWME6A1BB9jQ62DMZBg%40mail.g...
Attachments:
[text/x-patch] v8-alt-0001-switch-explain-to-unaligned-for-json-xml-yaml.patch (12.9K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/2-v8-alt-0001-switch-explain-to-unaligned-for-json-xml-yaml.patch)
download | inline diff:
From 103dfc1c56dfb1e95fb13c2c99da175e8a8a34e6 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v8-alt 1/7] switch explain to unaligned for json/xml/yaml
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.43.0
[text/x-patch] v8-alt-0002-separate-shared-instrumentation-initializatio.patch (22.7K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/3-v8-alt-0002-separate-shared-instrumentation-initializatio.patch)
download | inline diff:
From c405c4ba033b93149b727ea6b7d7a6e3e717a8b9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 16:10:06 -0400
Subject: [PATCH v8-alt 2/7] separate shared instrumentation initialization
---
src/backend/access/index/indexam.c | 51 ++---------
src/backend/executor/execParallel.c | 40 ++++++---
src/backend/executor/nodeBitmapIndexscan.c | 10 ++-
src/backend/executor/nodeIndexonlyscan.c | 99 ++++++++++++----------
src/backend/executor/nodeIndexscan.c | 99 ++++++++++++----------
src/include/access/genam.h | 7 +-
src/include/access/relscan.h | 1 -
src/include/executor/execParallel.h | 8 ++
src/include/executor/nodeIndexonlyscan.h | 6 ++
src/include/executor/nodeIndexscan.h | 6 ++
10 files changed, 174 insertions(+), 153 deletions(-)
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 23288a4f994..42e74b2442d 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -465,41 +465,26 @@ index_restrpos(IndexScanDesc scan)
/*
* index_parallelscan_estimate - estimate shared memory for parallel scan
*
- * When instrument=true, estimate includes SharedIndexScanInstrumentation
- * space. When parallel_aware=true, estimate includes whatever space the
- * index AM's amestimateparallelscan routine requested when called.
+ * Estimates the space needed for the ParallelIndexScanDesc, including the
+ * snapshot and any AM-specific parallel scan state.
*/
Size
index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
- Snapshot snapshot, bool instrument,
- bool parallel_aware, int nworkers)
+ Snapshot snapshot)
{
Size nbytes;
- Assert(instrument || parallel_aware);
-
RELATION_CHECKS;
nbytes = offsetof(ParallelIndexScanDescData, ps_snapshot_data);
nbytes = add_size(nbytes, EstimateSnapshotSpace(snapshot));
nbytes = MAXALIGN(nbytes);
- if (instrument)
- {
- Size sharedinfosz;
-
- sharedinfosz = offsetof(SharedIndexScanInstrumentation, winstrument) +
- nworkers * sizeof(IndexScanInstrumentation);
- nbytes = add_size(nbytes, sharedinfosz);
- nbytes = MAXALIGN(nbytes);
- }
-
/*
* If parallel scan index AM interface can't be used (or index AM provides
* no such interface), assume there is no AM-specific data needed
*/
- if (parallel_aware &&
- indexRelation->rd_indam->amestimateparallelscan != NULL)
+ if (indexRelation->rd_indam->amestimateparallelscan != NULL)
nbytes = add_size(nbytes,
indexRelation->rd_indam->amestimateparallelscan(indexRelation,
nkeys,
@@ -511,7 +496,7 @@ index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
/*
* index_parallelscan_initialize - initialize parallel scan
*
- * We initialize both the ParallelIndexScanDesc proper and the AM-specific
+ * We initialize the ParallelIndexScanDesc proper and the AM-specific
* information which follows it.
*
* This function calls access method specific initialization routine to
@@ -520,15 +505,11 @@ index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
*/
void
index_parallelscan_initialize(Relation heapRelation, Relation indexRelation,
- Snapshot snapshot, bool instrument,
- bool parallel_aware, int nworkers,
- SharedIndexScanInstrumentation **sharedinfo,
+ Snapshot snapshot,
ParallelIndexScanDesc target)
{
Size offset;
- Assert(instrument || parallel_aware);
-
RELATION_CHECKS;
offset = add_size(offsetof(ParallelIndexScanDescData, ps_snapshot_data),
@@ -537,29 +518,11 @@ index_parallelscan_initialize(Relation heapRelation, Relation indexRelation,
target->ps_locator = heapRelation->rd_locator;
target->ps_indexlocator = indexRelation->rd_locator;
- target->ps_offset_ins = 0;
target->ps_offset_am = 0;
SerializeSnapshot(snapshot, target->ps_snapshot_data);
- if (instrument)
- {
- Size sharedinfosz;
-
- target->ps_offset_ins = offset;
- sharedinfosz = offsetof(SharedIndexScanInstrumentation, winstrument) +
- nworkers * sizeof(IndexScanInstrumentation);
- offset = add_size(offset, sharedinfosz);
- offset = MAXALIGN(offset);
-
- /* Set leader's *sharedinfo pointer, and initialize stats */
- *sharedinfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(target, target->ps_offset_ins);
- memset(*sharedinfo, 0, sharedinfosz);
- (*sharedinfo)->num_workers = nworkers;
- }
-
/* aminitparallelscan is optional; assume no-op if not provided by AM */
- if (parallel_aware && indexRelation->rd_indam->aminitparallelscan != NULL)
+ if (indexRelation->rd_indam->aminitparallelscan != NULL)
{
void *amtarget;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 755191b51ef..10d2909892c 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -255,14 +255,20 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
e->pcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanEstimate((IndexScanState *) planstate,
+ e->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanEstimate((IndexScanState *) planstate,
- e->pcxt);
+ ExecIndexScanInstrumentEstimate((IndexScanState *) planstate,
+ e->pcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
- e->pcxt);
+ ExecIndexOnlyScanInstrumentEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -489,13 +495,20 @@ ExecParallelInitializeDSM(PlanState *planstate,
d->pcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanInitializeDSM((IndexScanState *) planstate,
+ d->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanInitializeDSM((IndexScanState *) planstate, d->pcxt);
+ ExecIndexScanInstrumentInitDSM((IndexScanState *) planstate,
+ d->pcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
- d->pcxt);
+ ExecIndexOnlyScanInstrumentInitDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -1367,13 +1380,20 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanInitializeWorker((IndexScanState *) planstate,
+ pwcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanInitializeWorker((IndexScanState *) planstate, pwcxt);
+ ExecIndexScanInstrumentInitWorker((IndexScanState *) planstate,
+ pwcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate,
+ pwcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate,
- pwcxt);
+ ExecIndexOnlyScanInstrumentInitWorker((IndexOnlyScanState *) planstate,
+ pwcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 70c55ee6d61..d0d78447f98 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -22,6 +22,7 @@
#include "postgres.h"
#include "access/genam.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeBitmapIndexscan.h"
@@ -394,7 +395,9 @@ ExecBitmapIndexScanInitializeDSM(BitmapIndexScanState *node,
node->biss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc,
size);
- shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id,
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
node->biss_SharedInfo);
/* Each per-worker area must start out as zeroes */
@@ -417,7 +420,10 @@ ExecBitmapIndexScanInitializeWorker(BitmapIndexScanState *node,
return;
node->biss_SharedInfo = (SharedIndexScanInstrumentation *)
- shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index de6154fd541..b76a1ea8110 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -36,6 +36,7 @@
#include "access/tupdesc.h"
#include "access/visibilitymap.h"
#include "catalog/pg_type.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeIndexonlyscan.h"
@@ -736,21 +737,11 @@ ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
- bool instrument = (node->ss.ps.instrument != NULL);
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- estate->es_snapshot,
- instrument, parallel_aware,
- pcxt->nworkers);
+ estate->es_snapshot);
shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -767,29 +758,14 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
index_parallelscan_initialize(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
estate->es_snapshot,
- instrument, parallel_aware, pcxt->nworkers,
- &node->ioss_SharedInfo, piscan);
+ piscan);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
- if (!parallel_aware)
- {
- /* Only here to initialize SharedInfo in DSM */
- return;
- }
-
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
@@ -837,27 +813,9 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- if (instrument)
- node->ioss_SharedInfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(piscan, piscan->ps_offset_ins);
-
- if (!parallel_aware)
- {
- /* Only here to set up worker node's SharedInfo */
- return;
- }
-
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
@@ -879,6 +837,57 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
}
+void
+ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ node->ioss_SharedInfo =
+ (SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->ioss_SharedInfo, 0, size);
+ node->ioss_SharedInfo->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->ioss_SharedInfo);
+}
+
+void
+ExecIndexOnlyScanInstrumentInitWorker(IndexOnlyScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->ioss_SharedInfo = (SharedIndexScanInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
/* ----------------------------------------------------------------
* ExecIndexOnlyScanRetrieveInstrumentation
*
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 1620d146071..9fa8f59d202 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -33,6 +33,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
#include "catalog/pg_am.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeIndexscan.h"
@@ -1674,21 +1675,11 @@ ExecIndexScanEstimate(IndexScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
node->iss_PscanLen = index_parallelscan_estimate(node->iss_RelationDesc,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- estate->es_snapshot,
- instrument, parallel_aware,
- pcxt->nworkers);
+ estate->es_snapshot);
shm_toc_estimate_chunk(&pcxt->estimator, node->iss_PscanLen);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -1705,29 +1696,14 @@ ExecIndexScanInitializeDSM(IndexScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_allocate(pcxt->toc, node->iss_PscanLen);
index_parallelscan_initialize(node->ss.ss_currentRelation,
node->iss_RelationDesc,
estate->es_snapshot,
- instrument, parallel_aware, pcxt->nworkers,
- &node->iss_SharedInfo, piscan);
+ piscan);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
- if (!parallel_aware)
- {
- /* Only here to initialize SharedInfo in DSM */
- return;
- }
-
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
@@ -1773,27 +1749,9 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- if (instrument)
- node->iss_SharedInfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(piscan, piscan->ps_offset_ins);
-
- if (!parallel_aware)
- {
- /* Only here to set up worker node's SharedInfo */
- return;
- }
-
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
@@ -1814,6 +1772,57 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
node->iss_OrderByKeys, node->iss_NumOrderByKeys);
}
+void
+ExecIndexScanInstrumentEstimate(IndexScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecIndexScanInstrumentInitDSM(IndexScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ node->iss_SharedInfo =
+ (SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->iss_SharedInfo, 0, size);
+ node->iss_SharedInfo->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->iss_SharedInfo);
+}
+
+void
+ExecIndexScanInstrumentInitWorker(IndexScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->iss_SharedInfo = (SharedIndexScanInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
/* ----------------------------------------------------------------
* ExecIndexScanRetrieveInstrumentation
*
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index b69320a7fc8..68bfe405db3 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -171,14 +171,9 @@ extern void index_endscan(IndexScanDesc scan);
extern void index_markpos(IndexScanDesc scan);
extern void index_restrpos(IndexScanDesc scan);
extern Size index_parallelscan_estimate(Relation indexRelation,
- int nkeys, int norderbys, Snapshot snapshot,
- bool instrument, bool parallel_aware,
- int nworkers);
+ int nkeys, int norderbys, Snapshot snapshot);
extern void index_parallelscan_initialize(Relation heapRelation,
Relation indexRelation, Snapshot snapshot,
- bool instrument, bool parallel_aware,
- int nworkers,
- SharedIndexScanInstrumentation **sharedinfo,
ParallelIndexScanDesc target);
extern void index_parallelrescan(IndexScanDesc scan);
extern IndexScanDesc index_beginscan_parallel(Relation heaprel,
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 960abf6c214..fd2076c582a 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -203,7 +203,6 @@ typedef struct ParallelIndexScanDescData
{
RelFileLocator ps_locator; /* physical table relation to scan */
RelFileLocator ps_indexlocator; /* physical index relation to scan */
- Size ps_offset_ins; /* Offset to SharedIndexScanInstrumentation */
Size ps_offset_am; /* Offset to am-specific structure */
char ps_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
} ParallelIndexScanDescData;
diff --git a/src/include/executor/execParallel.h b/src/include/executor/execParallel.h
index 5a2034811d5..f889034a5f5 100644
--- a/src/include/executor/execParallel.h
+++ b/src/include/executor/execParallel.h
@@ -19,6 +19,14 @@
#include "nodes/plannodes.h"
#include "utils/dsa.h"
+/*
+ * Offset added to plan_node_id to create a second TOC key for per-worker scan
+ * instrumentation. This allows scan nodes to store instrumentation in a
+ * separate DSM allocation from their parallel scan descriptor, avoiding the
+ * need to pack both into a single allocation with offset arithmetic.
+ */
+#define PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET UINT64CONST(0xD000000000000000)
+
typedef struct SharedExecutorInstrumentation SharedExecutorInstrumentation;
typedef struct ParallelExecutorInfo
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index ba807b8d8d9..b686244eb91 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -32,6 +32,12 @@ extern void ExecIndexOnlyScanReInitializeDSM(IndexOnlyScanState *node,
ParallelContext *pcxt);
extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInstrumentInitWorker(IndexOnlyScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecIndexOnlyScanRetrieveInstrumentation(IndexOnlyScanState *node);
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/executor/nodeIndexscan.h b/src/include/executor/nodeIndexscan.h
index 803be5b08df..c60cbc9a94b 100644
--- a/src/include/executor/nodeIndexscan.h
+++ b/src/include/executor/nodeIndexscan.h
@@ -28,6 +28,12 @@ extern void ExecIndexScanInitializeDSM(IndexScanState *node, ParallelContext *pc
extern void ExecIndexScanReInitializeDSM(IndexScanState *node, ParallelContext *pcxt);
extern void ExecIndexScanInitializeWorker(IndexScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecIndexScanInstrumentEstimate(IndexScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexScanInstrumentInitDSM(IndexScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexScanInstrumentInitWorker(IndexScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecIndexScanRetrieveInstrumentation(IndexScanState *node);
/*
--
2.43.0
[text/x-patch] v8-alt-0003-fix-bhs-non-parallel-aware-explain-analyze.patch (8.3K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/4-v8-alt-0003-fix-bhs-non-parallel-aware-explain-analyze.patch)
download | inline diff:
From afdeaf12ab7038574912e9469f4ad97ab1d3c83e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 16:27:22 -0400
Subject: [PATCH v8-alt 3/7] fix bhs non-parallel-aware explain analyze
---
src/backend/commands/explain.c | 2 +-
src/backend/executor/execParallel.c | 9 ++
src/backend/executor/nodeBitmapHeapscan.c | 100 ++++++++++++----------
src/include/executor/nodeBitmapHeapscan.h | 6 ++
4 files changed, 72 insertions(+), 45 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e4b70166b0e..8275bb2af61 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3949,7 +3949,7 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
/* Display stats for each parallel worker */
- if (planstate->pstate != NULL)
+ if (planstate->sinstrument != NULL)
{
for (int n = 0; n < planstate->sinstrument->num_workers; n++)
{
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 10d2909892c..8d5c69ec85e 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -299,6 +299,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecBitmapHeapEstimate((BitmapHeapScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentEstimate((BitmapHeapScanState *) planstate,
+ e->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -538,6 +541,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecBitmapHeapInitializeDSM((BitmapHeapScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentInitDSM((BitmapHeapScanState *) planstate,
+ d->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -1423,6 +1429,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecBitmapHeapInitializeWorker((BitmapHeapScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentInitWorker((BitmapHeapScanState *) planstate,
+ pwcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 73831aed451..4b1bff61214 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -38,6 +38,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
#include "access/visibilitymap.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
#include "executor/instrument.h"
#include "executor/nodeBitmapHeapscan.h"
@@ -495,18 +496,8 @@ void
ExecBitmapHeapEstimate(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
- Size size;
-
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
-
- /* account for instrumentation, if required */
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- {
- size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
- size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
- }
-
- shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ MAXALIGN(sizeof(ParallelBitmapHeapState)));
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -521,27 +512,14 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
ParallelBitmapHeapState *pstate;
- SharedBitmapHeapInstrumentation *sinstrument = NULL;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
- char *ptr;
- Size size;
/* If there's no DSA, there are no workers; initialize nothing. */
if (dsa == NULL)
return;
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- {
- size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
- size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
- }
-
- ptr = shm_toc_allocate(pcxt->toc, size);
- pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ pstate = (ParallelBitmapHeapState *)
+ shm_toc_allocate(pcxt->toc, MAXALIGN(sizeof(ParallelBitmapHeapState)));
pstate->tbmiterator = 0;
@@ -551,18 +529,8 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ConditionVariableInit(&pstate->cv);
- if (sinstrument)
- {
- sinstrument->num_workers = pcxt->nworkers;
-
- /* ensure any unfilled slots will contain zeroes */
- memset(sinstrument->sinstrument, 0,
- pcxt->nworkers * sizeof(BitmapHeapScanInstrumentation));
- }
-
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pstate);
node->pstate = pstate;
- node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -600,17 +568,61 @@ void
ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
- char *ptr;
-
Assert(node->ss.ps.state->es_query_dsa != NULL);
- ptr = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ node->pstate = (ParallelBitmapHeapState *)
+ shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+}
+
+void
+ExecBitmapHeapInstrumentEstimate(BitmapHeapScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedBitmapHeapInstrumentation, sinstrument) +
+ pcxt->nworkers * sizeof(BitmapHeapScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
- node->pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
+void
+ExecBitmapHeapInstrumentInitDSM(BitmapHeapScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
- if (node->ss.ps.instrument)
- node->sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ size = offsetof(SharedBitmapHeapInstrumentation, sinstrument) +
+ pcxt->nworkers * sizeof(BitmapHeapScanInstrumentation);
+ node->sinstrument =
+ (SharedBitmapHeapInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->sinstrument, 0, size);
+ node->sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->sinstrument);
+}
+
+void
+ExecBitmapHeapInstrumentInitWorker(BitmapHeapScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->sinstrument = (SharedBitmapHeapInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index 5d82f71abff..5c0b6592a1f 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,6 +28,12 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecBitmapHeapInstrumentEstimate(BitmapHeapScanState *node,
+ ParallelContext *pcxt);
+extern void ExecBitmapHeapInstrumentInitDSM(BitmapHeapScanState *node,
+ ParallelContext *pcxt);
+extern void ExecBitmapHeapInstrumentInitWorker(BitmapHeapScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecBitmapHeapRetrieveInstrumentation(BitmapHeapScanState *node);
#endif /* NODEBITMAPHEAPSCAN_H */
--
2.43.0
[text/x-patch] v8-alt-0004-Add-EXPLAIN-IO-infrastructure-and-BitmapHeapS.patch (56.4K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/5-v8-alt-0004-Add-EXPLAIN-IO-infrastructure-and-BitmapHeapS.patch)
download | inline diff:
From f8d911b6e1593aadba7a669de1ccc0384c50e0cb Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:17:22 -0400
Subject: [PATCH v8-alt 4/7] Add EXPLAIN (IO) infrastructure and BitmapHeapScan
IO instrumentation
---
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/heap/heapam.c | 10 ++
src/backend/commands/explain.c | 110 ++++++++++++++++++
src/backend/commands/explain_state.c | 15 +++
src/backend/executor/nodeBitmapHeapscan.c | 19 ++-
src/backend/storage/aio/read_stream.c | 87 ++++++++++++++
src/include/access/relscan.h | 7 ++
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument_node.h | 50 ++++++++
src/include/storage/read_stream.h | 2 +
.../modules/index/expected/killtuples.out | 44 +++----
src/test/modules/index/specs/killtuples.spec | 2 +-
src/test/regress/expected/explain.out | 16 +--
.../regress/expected/incremental_sort.out | 4 +-
src/test/regress/expected/memoize.out | 2 +-
src/test/regress/expected/merge.out | 2 +-
src/test/regress/expected/partition_prune.out | 2 +-
src/test/regress/expected/select_parallel.out | 6 +-
src/test/regress/expected/subselect.out | 2 +-
src/test/regress/sql/explain.sql | 16 +--
src/test/regress/sql/incremental_sort.sql | 4 +-
src/test/regress/sql/memoize.sql | 2 +-
src/test/regress/sql/merge.sql | 2 +-
src/test/regress/sql/partition_prune.sql | 2 +-
src/test/regress/sql/select_parallel.sql | 6 +-
src/test/regress/sql/subselect.sql | 2 +-
src/tools/pgindent/typedefs.list | 2 +
28 files changed, 373 insertions(+), 59 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..4951b624893 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>TRUE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e06ce2db2cf..721f6f3ffd9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -43,6 +43,7 @@
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/instrument_node.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/lmgr.h"
@@ -1199,6 +1200,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1311,6 +1313,14 @@ heap_beginscan(Relation relation, Snapshot snapshot,
sizeof(TBMIterateResult));
}
+ /* enable read stream instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ {
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ read_stream_enable_stats(scan->rs_read_stream,
+ &scan->rs_base.rs_instrument->io);
+ }
+
scan->rs_vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 8275bb2af61..0fab21a5cfa 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,7 @@
*/
#include "postgres.h"
+#include "access/relscan.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +140,8 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -2009,6 +2012,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -3985,6 +3989,112 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f inprogress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats = {0};
+
+ if (!es->io)
+ return;
+
+ /*
+ * Initialize counters with stats from the local process first.
+ *
+ * The scan descriptor may not exist, e.g. if the scan did not start,
+ * or because of debug_parallel_query=regress. We still want to collect
+ * data from workers.
+ */
+ if (planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument)
+ {
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+ }
+
+ /*
+ * Accumulate data from parallel workers (if any).
+ */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 77f59b8e500..3d5f92bfa91 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -79,6 +79,7 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
ListCell *lc;
bool timing_set = false;
bool buffers_set = false;
+ bool io_set = false;
bool summary_set = false;
/* Parse options list. */
@@ -159,6 +160,11 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ {
+ io_set = true;
+ es->io = defGetBoolean(opt);
+ }
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -179,12 +185,21 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
/* if the buffers was not set explicitly, set default value */
es->buffers = (buffers_set) ? es->buffers : es->analyze;
+ /* if the IO was not set explicitly, set default value */
+ es->io = (io_set) ? es->io : es->analyze;
+
/* check that timing is used with EXPLAIN ANALYZE */
if (es->timing && !es->analyze)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 4b1bff61214..5d1cb727e20 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -145,13 +145,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -331,6 +338,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect prefetch info for this process from the read_stream */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 0b6cdf7c873..936e08ea450 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -74,6 +74,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -123,6 +124,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -188,6 +192,73 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+/*
+ * Update stream stats about waits for I/O when consuming buffers.
+ *
+ * We count the number of I/O waits while pulling buffers out of a stream.
+ */
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
+/*
+ * Enable collection of stats into the provided IOStats.
+ */
+void
+read_stream_enable_stats(ReadStream *stream, IOStats *stats)
+{
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = stream->max_pinned_buffers;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -426,6 +497,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -1021,6 +1095,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -1036,6 +1111,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
* to avoid having to effectively do another synchronous IO for
* the next block (if it were also a miss).
*/
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -1100,6 +1181,10 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+ /* Count it as a wait if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
+
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
if (++stream->oldest_io_index == stream->max_ios)
@@ -1228,6 +1313,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fd2076c582a..8df293c68d4 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -25,6 +25,8 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -64,6 +66,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 4647785fd35..9e3d7186868 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -69,6 +69,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 5a48bc6fbb1..b412f00f70c 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 2a0ff377a73..56a695304dd 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -40,6 +40,55 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
@@ -71,6 +120,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..aebb1fafb31 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -103,5 +104,6 @@ extern BlockNumber read_stream_pause(ReadStream *stream);
extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern void read_stream_enable_stats(ReadStream *stream, IOStats *stats);
#endif /* READ_STREAM_H */
diff --git a/src/test/modules/index/expected/killtuples.out b/src/test/modules/index/expected/killtuples.out
index a3db2c40936..550453fece8 100644
--- a/src/test/modules/index/expected/killtuples.out
+++ b/src/test/modules/index/expected/killtuples.out
@@ -8,7 +8,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -24,7 +24,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -42,7 +42,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -58,7 +58,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_btree on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -84,7 +84,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -100,7 +100,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -118,7 +118,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -134,7 +134,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -161,7 +161,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -177,7 +177,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -195,7 +195,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -211,7 +211,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_gist on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -237,7 +237,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -253,7 +253,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=1.00 loops=1)
@@ -271,7 +271,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -287,7 +287,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -312,7 +312,7 @@ step flush: SELECT FROM pg_stat_force_next_flush();
step disable_seq: SET enable_seqscan = false;
step disable_bitmap: SET enable_bitmapscan = false;
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -328,7 +328,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
---------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=408.00 loops=1)
@@ -346,7 +346,7 @@ t
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -362,7 +362,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
-------------------------------------------------------------------------------------
Index Scan using kill_prior_tuple_hash on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -389,7 +389,7 @@ step disable_seq: SET enable_seqscan = false;
step delete: DELETE FROM kill_prior_tuple;
step flush: SELECT FROM pg_stat_force_next_flush();
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
@@ -408,7 +408,7 @@ t
(1 row)
step measure: UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_blks_hit FROM pg_statio_all_tables WHERE relname = 'kill_prior_tuple');
-step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
+step access: EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1;
QUERY PLAN
--------------------------------------------------------------------------
Bitmap Heap Scan on kill_prior_tuple (actual rows=0.00 loops=1)
diff --git a/src/test/modules/index/specs/killtuples.spec b/src/test/modules/index/specs/killtuples.spec
index 3b98ff9f76d..a5d5c33d788 100644
--- a/src/test/modules/index/specs/killtuples.spec
+++ b/src/test/modules/index/specs/killtuples.spec
@@ -45,7 +45,7 @@ step measure { UPDATE counter SET heap_accesses = (SELECT heap_blks_read + heap_
step result { SELECT ((heap_blks_read + heap_blks_hit - counter.heap_accesses) > 0) AS has_new_heap_accesses FROM counter, pg_statio_all_tables WHERE relname = 'kill_prior_tuple'; }
-step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
+step access { EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF) SELECT * FROM kill_prior_tuple WHERE key = 1; }
step delete { DELETE FROM kill_prior_tuple; }
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..d7754c6f2cb 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -68,7 +68,7 @@ select explain_filter('explain select * from int8_tbl i8');
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -76,7 +76,7 @@ select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8'
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
explain_filter
--------------------------------------------------------------------------------------------------------
Seq Scan on public.int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -392,7 +392,7 @@ select explain_filter('explain (memory) select * from int8_tbl i8');
Memory: used=NkB allocated=NkB
(2 rows)
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -740,7 +740,7 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
(3 rows)
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -768,7 +768,7 @@ select explain_filter('explain (analyze,serialize binary,buffers,timing) select
(4 rows)
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -778,7 +778,7 @@ select explain_filter('explain (analyze,buffers off,serialize) create temp table
(4 rows)
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -791,7 +791,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
explain_filter
----------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
@@ -803,7 +803,7 @@ select explain_filter('explain (analyze,buffers off,costs off) select sum(n) ove
(6 rows)
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
explain_filter
----------------------------------------------------------------------------------------
WindowAgg (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/incremental_sort.out b/src/test/regress/expected/incremental_sort.out
index 1e6e020fea8..55d5a52d96b 100644
--- a/src/test/regress/expected/incremental_sort.out
+++ b/src/test/regress/expected/incremental_sort.out
@@ -39,7 +39,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -55,7 +55,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/expected/memoize.out b/src/test/regress/expected/memoize.out
index 218972dfab8..950c0fbae69 100644
--- a/src/test/regress/expected/memoize.out
+++ b/src/test/regress/expected/memoize.out
@@ -10,7 +10,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
index 9cb1d87066a..f1331c307d3 100644
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -1623,7 +1623,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index deacdd75807..1012be91689 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/expected/select_parallel.out b/src/test/regress/expected/select_parallel.out
index 933921d1860..bfaf10a2906 100644
--- a/src/test/regress/expected/select_parallel.out
+++ b/src/test/regress/expected/select_parallel.out
@@ -580,7 +580,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
QUERY PLAN
@@ -606,7 +606,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -1170,7 +1170,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
QUERY PLAN
----------------------------------------------------------------
Gather (actual rows=10000.00 loops=1)
diff --git a/src/test/regress/expected/subselect.out b/src/test/regress/expected/subselect.out
index 200236a0a69..0a1de6d7259 100644
--- a/src/test/regress/expected/subselect.out
+++ b/src/test/regress/expected/subselect.out
@@ -2051,7 +2051,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..dbd210a07f2 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -63,8 +63,8 @@ set track_io_timing = off;
explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
@@ -105,7 +105,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
-- MEMORY option
select explain_filter('explain (memory) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, buffers off) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
@@ -177,17 +177,17 @@ select explain_filter('explain (verbose) declare test_cur cursor for select * fr
select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
-select explain_filter('explain (analyze,buffers off,serialize) select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
-select explain_filter('explain (analyze,buffers off,serialize) create temp table explain_temp as select * from int8_tbl i8');
+select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
-- Test tuplestore storage usage in Window aggregate (memory case)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,10) a(n)');
-- Test tuplestore storage usage in Window aggregate (disk case)
set work_mem to 64;
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over() from generate_series(1,2500) a(n)');
-- Test tuplestore storage usage in Window aggregate (memory and disk case, final result is disk)
-select explain_filter('explain (analyze,buffers off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
+select explain_filter('explain (analyze,buffers off,io off,costs off) select sum(n) over(partition by m) from (SELECT n < 3 as m, n from generate_series(1,2500) a(n))');
reset work_mem;
diff --git a/src/test/regress/sql/incremental_sort.sql b/src/test/regress/sql/incremental_sort.sql
index bbe658a7588..1c8036faade 100644
--- a/src/test/regress/sql/incremental_sort.sql
+++ b/src/test/regress/sql/incremental_sort.sql
@@ -21,7 +21,7 @@ declare
line text;
begin
for line in
- execute 'explain (analyze, costs off, summary off, timing off, buffers off) ' || query
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off) ' || query
loop
out_line := regexp_replace(line, '\d+kB', 'NNkB', 'g');
return next;
@@ -38,7 +38,7 @@ declare
element jsonb;
matching_nodes jsonb := '[]'::jsonb;
begin
- execute 'explain (analyze, costs off, summary off, timing off, buffers off, format ''json'') ' || query into strict elements;
+ execute 'explain (analyze, costs off, summary off, timing off, buffers off, io off, format ''json'') ' || query into strict elements;
while jsonb_array_length(elements) > 0 loop
element := elements->0;
elements := elements - 0;
diff --git a/src/test/regress/sql/memoize.sql b/src/test/regress/sql/memoize.sql
index e39bbb65391..532df97ee90 100644
--- a/src/test/regress/sql/memoize.sql
+++ b/src/test/regress/sql/memoize.sql
@@ -11,7 +11,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
if hide_hitmiss = true then
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
index 2660b19f238..c82e194078e 100644
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -1074,7 +1074,7 @@ $$
DECLARE ln text;
BEGIN
FOR ln IN
- EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off) ' ||
+ EXECUTE 'explain (analyze, timing off, summary off, costs off, buffers off, io off) ' ||
query
LOOP
ln := regexp_replace(ln, '(Memory( Usage)?|Buckets|Batches): \S*', '\1: xxx', 'g');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d93c0c03bab..212de0e6285 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -12,7 +12,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
query)
loop
ln := regexp_replace(ln, 'Maximum Storage: \d+', 'Maximum Storage: N');
diff --git a/src/test/regress/sql/select_parallel.sql b/src/test/regress/sql/select_parallel.sql
index 71a75bc86ea..56e1ceac492 100644
--- a/src/test/regress/sql/select_parallel.sql
+++ b/src/test/regress/sql/select_parallel.sql
@@ -230,7 +230,7 @@ select count(*) from bmscantest where a>1;
-- test accumulation of stats for parallel nodes
reset enable_seqscan;
alter table tenk2 set (parallel_workers = 0);
-explain (analyze, timing off, summary off, costs off, buffers off)
+explain (analyze, timing off, summary off, costs off, buffers off, io off)
select count(*) from tenk1, tenk2 where tenk1.hundred > 1
and tenk2.thousand=0;
alter table tenk2 reset (parallel_workers);
@@ -242,7 +242,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, timing off, summary off, costs off, buffers off)
+ explain (analyze, timing off, summary off, costs off, buffers off, io off)
select * from
(select ten from tenk1 where ten < 100 order by ten) ss
right join (values (1),(2),(3)) v(x) on true
@@ -450,7 +450,7 @@ explain (costs off)
-- to increase the parallel query test coverage
SAVEPOINT settings;
SET LOCAL debug_parallel_query = 1;
-EXPLAIN (analyze, timing off, summary off, costs off, buffers off) SELECT * FROM tenk1;
+EXPLAIN (analyze, timing off, summary off, costs off, buffers off, io off) SELECT * FROM tenk1;
ROLLBACK TO SAVEPOINT settings;
-- provoke error in worker
diff --git a/src/test/regress/sql/subselect.sql b/src/test/regress/sql/subselect.sql
index 4cd016f4ac3..3e930d8b56d 100644
--- a/src/test/regress/sql/subselect.sql
+++ b/src/test/regress/sql/subselect.sql
@@ -1016,7 +1016,7 @@ $$
declare ln text;
begin
for ln in
- explain (analyze, summary off, timing off, costs off, buffers off)
+ explain (analyze, summary off, timing off, costs off, buffers off, io off)
select * from (select pk,c2 from sq_limit order by c1,pk) as x limit 3
loop
ln := regexp_replace(ln, 'Memory: \S*', 'Memory: xxx');
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0c5493bd47f..fce46f79ac2 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1276,6 +1276,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3115,6 +3116,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.43.0
[text/x-patch] v8-alt-0005-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch (40.0K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/6-v8-alt-0005-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch)
download | inline diff:
From 6a6bfb5ae89ae2399431f77a03d79961ded6d2df Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:28:29 -0400
Subject: [PATCH v8-alt 5/7] Add EXPLAIN (IO) instrumentation for SeqScan
---
.../postgres_fdw/expected/postgres_fdw.out | 8 +-
contrib/postgres_fdw/sql/postgres_fdw.sql | 8 +-
src/backend/commands/explain.c | 25 ++++
src/backend/executor/execParallel.c | 11 ++
src/backend/executor/nodeSeqscan.c | 116 ++++++++++++++++--
src/include/executor/instrument_node.h | 19 +++
src/include/executor/nodeSeqscan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/test/isolation/expected/merge-update.out | 2 +-
src/test/isolation/specs/merge-update.spec | 5 +-
src/test/regress/expected/explain.out | 26 +++-
src/test/regress/expected/partition_prune.out | 34 ++---
src/test/regress/sql/explain.sql | 12 +-
src/test/regress/sql/partition_prune.sql | 34 ++---
src/tools/pgindent/typedefs.list | 2 +
15 files changed, 247 insertions(+), 65 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index cd22553236f..6b460a90158 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -11980,7 +11980,7 @@ SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c
Filter: (async_pt_3.a = local_tbl.a)
(15 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
QUERY PLAN
----------------------------------------------------------------------------------
@@ -12226,7 +12226,7 @@ SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt W
Remote SQL: SELECT a, b, c FROM public.base_tbl2 WHERE ((a < 3000))
(20 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
QUERY PLAN
--------------------------------------------------------------------------------------------
@@ -12270,7 +12270,7 @@ SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
Filter: (t1_3.b === 505)
(14 rows)
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------
@@ -12430,7 +12430,7 @@ DELETE FROM async_pt WHERE b = 0 RETURNING *;
DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
QUERY PLAN
----------------------------------------------------------------------------
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 59963e298b8..e8b96a16849 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -4093,7 +4093,7 @@ ALTER FOREIGN TABLE async_p2 OPTIONS (use_remote_estimate 'true');
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
SELECT * FROM local_tbl, async_pt WHERE local_tbl.a = async_pt.a AND local_tbl.c = 'bar';
@@ -4168,13 +4168,13 @@ ANALYZE local_tbl;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
SELECT * FROM local_tbl t1 LEFT JOIN (SELECT *, (SELECT count(*) FROM async_pt WHERE a < 3000) FROM async_pt WHERE a < 3000) t2 ON t1.a = t2.a;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
SELECT * FROM async_pt t1 WHERE t1.b === 505 LIMIT 1;
@@ -4226,7 +4226,7 @@ DELETE FROM async_p1;
DELETE FROM async_p2;
DELETE FROM async_p3;
-EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
+EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF, IO OFF)
SELECT * FROM async_pt;
-- Clean up
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 0fab21a5cfa..56c749c0d86 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2031,6 +2031,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -4085,6 +4086,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 8d5c69ec85e..8c2ee4cd463 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -253,6 +253,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecSeqScanEstimate((SeqScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentEstimate((SeqScanState *) planstate,
+ e->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -496,6 +499,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeDSM((SeqScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitDSM((SeqScanState *) planstate,
+ d->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -1144,6 +1150,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
@@ -1384,6 +1393,8 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
case T_SeqScanState:
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..d169cb6b3c2 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -29,6 +29,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
+#include "executor/execParallel.h"
#include "executor/execScan.h"
#include "executor/executor.h"
#include "executor/nodeSeqscan.h"
@@ -65,15 +66,21 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
- 0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ 0, NULL, flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -297,6 +304,24 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * Collect IO stats for this process into shared instrumentation.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber <= node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
/*
* get information from node
*/
@@ -370,6 +395,13 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -378,9 +410,7 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
}
/* ----------------------------------------------------------------
@@ -410,10 +440,78 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+}
+
+void
+ExecSeqScanInstrumentEstimate(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ offsetof(SharedSeqScanInstrumentation, sinstrument) +
+ sizeof(SeqScanInstrumentation) * pcxt->nworkers);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecSeqScanInstrumentInitDSM(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedSeqScanInstrumentation *sinstrument;
+ Size size;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument) +
+ sizeof(SeqScanInstrumentation) * pcxt->nworkers;
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->sinstrument = sinstrument;
+}
+
+void
+ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.state->es_instrument)
+ return;
+
+ node->sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 56a695304dd..b7226df6840 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -258,4 +258,23 @@ typedef struct SharedIncrementalSortInfo
IncrementalSortInfo sinfo[FLEXIBLE_ARRAY_MEMBER];
} SharedIncrementalSortInfo;
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..9c0ad4879d7 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -28,4 +28,13 @@ extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecSeqScanInstrumentEstimate(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitDSM(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
+
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 090cfccf65f..ee9ca731f70 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1666,6 +1666,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ struct SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index 821565b4303..dd9c67c4dc1 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -358,7 +358,7 @@ step pa_merge1:
step explain_pa_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index b902779edd6..cf387eccefc 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -33,6 +33,9 @@ setup
FOR ln IN EXECUTE $1 LOOP
-- Ignore hash memory usage because it varies depending on the system
CONTINUE WHEN (ln ~ 'Memory Usage');
+ -- Ignore prefetch and I/O because it varies depending on the system
+ CONTINUE WHEN (ln ~ 'Prefetch');
+ CONTINUE WHEN (ln ~ 'I/O');
RETURN NEXT ln;
END LOOP;
END;
@@ -165,7 +168,7 @@ step "pa_merge2a"
step "explain_pa_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO pa_target t
USING (SELECT 1 as key, 'pa_merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index d7754c6f2cb..3e504911db4 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -85,7 +85,7 @@ select explain_filter('explain (analyze, buffers off, io off, verbose) select *
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
@@ -286,7 +300,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
explain_filter
-------------------------------------
[ +
@@ -421,7 +435,7 @@ select explain_filter('explain (memory, summary, format yaml) select * from int8
Planning Time: N.N
(1 row)
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
explain_filter
------------------------------------
[ +
@@ -524,7 +538,7 @@ set parallel_tuple_cost=0;
set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -749,7 +763,7 @@ select explain_filter('explain (analyze,buffers off,io off,serialize) select * f
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
explain_filter
-----------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual rows=N.N loops=N)
@@ -758,7 +772,7 @@ select explain_filter('explain (analyze,serialize text,buffers,timing off) selec
Execution Time: N.N ms
(4 rows)
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
explain_filter
-------------------------------------------------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N) (actual time=N.N..N.N rows=N.N loops=N)
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 1012be91689..6bd15f26aec 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2315,7 +2315,7 @@ begin;
-- Test run-time pruning using stable functions
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=1.00 loops=1)
@@ -2325,7 +2325,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(4 rows)
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=4.00 loops=1)
@@ -2340,7 +2340,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) select * from
(9 rows)
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
QUERY PLAN
---------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2376,7 +2376,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -2687,7 +2687,7 @@ reset parallel_tuple_cost;
reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
QUERY PLAN
----------------------------------------------------------------------------
@@ -2864,7 +2864,7 @@ union all
select tableoid::regclass,a,b from ab
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
QUERY PLAN
-------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3019,7 +3019,7 @@ create index tprt6_idx on tprt_6 (col1);
insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3046,7 +3046,7 @@ select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3097,7 +3097,7 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3124,7 +3124,7 @@ select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
Index Searches: 0
(21 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3194,7 +3194,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
QUERY PLAN
-----------------------------------------------------------------------------
@@ -3232,7 +3232,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
QUERY PLAN
-------------------------------------------------------------------
@@ -3478,7 +3478,7 @@ create table mc3p1 partition of mc3p
create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
QUERY PLAN
-----------------------------------------------------------
@@ -3498,7 +3498,7 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
QUERY PLAN
------------------------------------------------------------------
@@ -3513,7 +3513,7 @@ execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
QUERY PLAN
-------------------------------------------------------------------
@@ -3535,7 +3535,7 @@ insert into boolvalues values('t'),('f');
create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
QUERY PLAN
--------------------------------------------------------------
@@ -3550,7 +3550,7 @@ select * from boolp where a = (select value from boolvalues where value);
Filter: (a = (InitPlan expr_1).col1)
(9 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
QUERY PLAN
--------------------------------------------------------------
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index dbd210a07f2..b7907a228fa 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -65,7 +65,7 @@ explain (costs off) select 1 as a, 2 as b having false;
select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, io off, verbose) select * from int8_tbl i8');
-select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
@@ -82,7 +82,7 @@ select explain_filter('explain verbose select sum(unique1) over w1, sum(unique2)
-- Check output including I/O timings. These fields are conditional
-- but always set in JSON format, so check them only in this case.
set track_io_timing = on;
-select explain_filter('explain (analyze, buffers, format json) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io off, format json) select * from int8_tbl i8');
set track_io_timing = off;
-- SETTINGS option
@@ -107,7 +107,7 @@ select explain_filter('explain (analyze, generic_plan) select unique1 from tenk1
select explain_filter('explain (memory) select * from int8_tbl i8');
select explain_filter('explain (memory, analyze, buffers off, io off) select * from int8_tbl i8');
select explain_filter('explain (memory, summary, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (memory, analyze, format json) select * from int8_tbl i8');
+select explain_filter('explain (memory, analyze, io off, format json) select * from int8_tbl i8');
prepare int8_query as select * from int8_tbl i8;
select explain_filter('explain (memory) execute int8_query');
@@ -147,7 +147,7 @@ set min_parallel_table_scan_size=0;
set max_parallel_workers_per_gather=4;
select jsonb_pretty(
- explain_filter_to_json('explain (analyze, verbose, buffers, format json)
+ explain_filter_to_json('explain (analyze, verbose, buffers, io off, format json)
select * from tenk1 order by tenthous')
-- remove "Workers" node of the Seq Scan plan node
#- '{0,Plan,Plans,0,Plans,0,Workers}'
@@ -178,8 +178,8 @@ select explain_filter('explain (verbose) create table test_ctas as select 1');
-- Test SERIALIZE option
select explain_filter('explain (analyze,buffers off,io off,serialize) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize text,buffers,timing off) select * from int8_tbl i8');
-select explain_filter('explain (analyze,serialize binary,buffers,timing) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize text,buffers,io off,timing off) select * from int8_tbl i8');
+select explain_filter('explain (analyze,serialize binary,buffers,io off,timing) select * from int8_tbl i8');
-- this tests an edge case where we have no data to return
select explain_filter('explain (analyze,buffers off,io off,serialize) create temp table explain_temp as select * from int8_tbl i8');
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 212de0e6285..c5968932aed 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -553,13 +553,13 @@ begin;
create function list_part_fn(int) returns int as $$ begin return $1; end;$$ language plpgsql stable;
-- Ensure pruning works using a stable function containing no Vars
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1);
-- Ensure pruning does not take place when the function has a Var parameter
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(a);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(a);
-- Ensure pruning does not take place when the expression contains a Var.
-explain (analyze, costs off, summary off, timing off, buffers off) select * from list_part where a = list_part_fn(1) + a;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from list_part where a = list_part_fn(1) + a;
rollback;
@@ -582,7 +582,7 @@ declare
ln text;
begin
for ln in
- execute format('explain (analyze, costs off, summary off, timing off, buffers off) %s',
+ execute format('explain (analyze, costs off, summary off, timing off, buffers off, io off) %s',
$1)
loop
ln := regexp_replace(ln, 'Workers Launched: \d+', 'Workers Launched: N');
@@ -669,7 +669,7 @@ reset min_parallel_table_scan_size;
reset max_parallel_workers_per_gather;
-- Test run-time partition pruning with an initplan
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
@@ -697,7 +697,7 @@ union all
) ab where a = $1 and b = (select -10);
-- Ensure the xy_1 subplan is not pruned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q6(1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q6(1);
-- Ensure we see just the xy_1 row.
execute ab_q6(100);
@@ -752,10 +752,10 @@ insert into tprt values (10), (20), (501), (502), (505), (1001), (4500);
set enable_hashjoin = off;
set enable_mergejoin = off;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -768,10 +768,10 @@ order by tbl1.col1, tprt.col1;
-- Multiple partitions
insert into tbl1 values (1001), (1010), (1011);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 > tprt.col1;
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 inner join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -785,7 +785,7 @@ order by tbl1.col1, tprt.col1;
-- Last partition
delete from tbl1;
insert into tbl1 values (4400);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 < tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -795,7 +795,7 @@ order by tbl1.col1, tprt.col1;
-- No matching partition
delete from tbl1;
insert into tbl1 values (10000);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from tbl1 join tprt on tbl1.col1 = tprt.col1;
select tbl1.col1, tprt.col1 from tbl1
@@ -917,7 +917,7 @@ create table mc3p2 partition of mc3p
for values from (2, minvalue, minvalue) to (3, maxvalue, maxvalue);
insert into mc3p values (0, 1, 1), (1, 1, 1), (2, 1, 1);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from mc3p where a < 3 and abs(b) = 1;
--
@@ -927,12 +927,12 @@ select * from mc3p where a < 3 and abs(b) = 1;
--
prepare ps1 as
select * from mc3p where a = $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps1(1);
deallocate ps1;
prepare ps2 as
select * from mc3p where a <= $1 and abs(b) < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
execute ps2(1);
deallocate ps2;
@@ -946,10 +946,10 @@ create table boolp (a bool) partition by list (a);
create table boolp_t partition of boolp for values in('t');
create table boolp_f partition of boolp for values in('f');
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where value);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from boolp where a = (select value from boolvalues where not value);
drop table boolp;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fce46f79ac2..0a9e5247634 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2793,6 +2793,7 @@ SelfJoinCandidate
SemTPadded
SemiAntiJoinFactors
SeqScan
+SeqScanInstrumentation
SeqScanState
SeqTable
SeqTableData
@@ -2857,6 +2858,7 @@ SharedMemoizeInfo
SharedRecordTableEntry
SharedRecordTableKey
SharedRecordTypmodRegistry
+SharedSeqScanInstrumentation
SharedSortInfo
SharedTuplestore
SharedTuplestoreAccessor
--
2.43.0
[text/x-patch] v8-alt-0006-Add-EXPLAIN-IO-instrumentation-for-TidRangeSc.patch (11.5K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/7-v8-alt-0006-Add-EXPLAIN-IO-instrumentation-for-TidRangeSc.patch)
download | inline diff:
From f58863dd70623ada4fc42950416701e540612619 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:31:18 -0400
Subject: [PATCH v8-alt 6/7] Add EXPLAIN (IO) instrumentation for TidRangeScan
---
src/backend/commands/explain.c | 25 ++++++
src/backend/executor/execParallel.c | 12 +++
src/backend/executor/nodeTidrangescan.c | 112 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 18 ++++
src/include/executor/nodeTidrangescan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 171 insertions(+), 8 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 56c749c0d86..9e228d98c72 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2148,6 +2148,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4110,6 +4111,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 8c2ee4cd463..45e955b260a 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -287,6 +287,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentEstimate((TidRangeScanState *) planstate,
+ e->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -532,6 +535,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitDSM((TidRangeScanState *) planstate,
+ d->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -1153,6 +1159,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
@@ -1426,6 +1435,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitWorker((TidRangeScanState *) planstate,
+ pwcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..88891974ce4 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -18,6 +18,7 @@
#include "access/sysattr.h"
#include "access/tableam.h"
#include "catalog/pg_operator.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
#include "executor/nodeTidrangescan.h"
#include "nodes/nodeFuncs.h"
@@ -242,12 +243,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +350,19 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /* Collect IO stats for this process into shared instrumentation */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber <= node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (scan && scan->rs_instrument)
+ AccumulateIOStats(&si->stats.io,
+ &scan->rs_instrument->io);
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -454,6 +475,13 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -462,9 +490,7 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -494,11 +520,81 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+}
+
+void
+ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ offsetof(SharedTidRangeScanInstrumentation, sinstrument) +
+ sizeof(TidRangeScanInstrumentation) * pcxt->nworkers);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedTidRangeScanInstrumentation *sinstrument;
+ Size size;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument) +
+ sizeof(TidRangeScanInstrumentation) * pcxt->nworkers;
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->trss_sinstrument = sinstrument;
+}
+
+void
+ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.state->es_instrument)
+ return;
+
+ node->trss_sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index b7226df6840..299d14ef341 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -277,4 +277,22 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+
+/*
+ * Instrumentation information for TID range scans
+ */
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..9e7d0a357bb 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -28,4 +28,13 @@ extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelConte
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
+
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index ee9ca731f70..a94ceaa7c58 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1918,6 +1918,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ struct SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0a9e5247634..0364fda4a86 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2860,6 +2860,7 @@ SharedRecordTableKey
SharedRecordTypmodRegistry
SharedSeqScanInstrumentation
SharedSortInfo
+SharedTidRangeScanInstrumentation
SharedTuplestore
SharedTuplestoreAccessor
SharedTuplestoreChunk
@@ -3158,6 +3159,7 @@ TidOpExpr
TidPath
TidRangePath
TidRangeScan
+TidRangeScanInstrumentation
TidRangeScanState
TidScan
TidScanState
--
2.43.0
[text/x-patch] v8-alt-0007-other-test-changes.patch (28.1K, ../../CAAKRu_Z4rM416JcKRx+rBz8JS9C-6FaLQnN25SeT6ui-VMntVA@mail.gmail.com/8-v8-alt-0007-other-test-changes.patch)
download | inline diff:
From 7875452c055207f7877082872997ca6af3f7a197 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:42:39 -0400
Subject: [PATCH v8-alt 7/7] other test changes?
these didn't seem needed for earlier commits to pass tests
---
src/test/isolation/expected/merge-update.out | 4 +-
src/test/isolation/specs/merge-update.spec | 2 +-
src/test/regress/expected/partition_prune.out | 60 +++++++++----------
src/test/regress/sql/partition_prune.sql | 60 +++++++++----------
4 files changed, 63 insertions(+), 63 deletions(-)
diff --git a/src/test/isolation/expected/merge-update.out b/src/test/isolation/expected/merge-update.out
index dd9c67c4dc1..a7578d8aea6 100644
--- a/src/test/isolation/expected/merge-update.out
+++ b/src/test/isolation/expected/merge-update.out
@@ -70,7 +70,7 @@ step merge1:
step c1: COMMIT;
step explain_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO target t
USING (SELECT 1 as key, 'merge2a' as val) s
ON s.key = t.key
@@ -175,7 +175,7 @@ step merge1:
step explain_merge2a:
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO target t
USING (SELECT 1 as key, 'merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/isolation/specs/merge-update.spec b/src/test/isolation/specs/merge-update.spec
index cf387eccefc..66d851a7aa1 100644
--- a/src/test/isolation/specs/merge-update.spec
+++ b/src/test/isolation/specs/merge-update.spec
@@ -119,7 +119,7 @@ step "merge2a"
step "explain_merge2a"
{
SELECT explain_filter($$
- EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
+ EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF, IO OFF)
MERGE INTO target t
USING (SELECT 1 as key, 'merge2a' as val) s
ON s.key = t.key
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 6bd15f26aec..811f1467bcf 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2169,7 +2169,7 @@ create table ab_a3_b3 partition of ab_a3 for values in (3);
set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2, 3);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2182,7 +2182,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1
Filter: ((a >= $1) AND (a <= $2) AND (b <= $3))
(8 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (1, 2, 3);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2205,7 +2205,7 @@ deallocate ab_q1;
-- Runtime pruning after optimizer pruning
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2216,7 +2216,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1
Filter: ((a >= $1) AND (a <= $2) AND (b < 3))
(6 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 4);
QUERY PLAN
------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2235,7 +2235,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q2 (2, 2);
QUERY PLAN
----------------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2253,7 +2253,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q3 (2, 2);
QUERY PLAN
----------------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -2755,7 +2755,7 @@ select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1
(61 rows)
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
QUERY PLAN
----------------------------------------------------------------------------------
@@ -2802,7 +2802,7 @@ select * from (select * from ab where a = 1 union all select * from ab) ab where
(40 rows)
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
QUERY PLAN
----------------------------------------------------------------------------------
@@ -3278,7 +3278,7 @@ alter table part_cab attach partition part_abc_p1 for values in(3);
prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute part_abc_q1 (1, 2, 3);
QUERY PLAN
-------------------------------------------------------------
Seq Scan on part_abc_p1 part_abc (actual rows=0.00 loops=1)
@@ -3303,7 +3303,7 @@ select * from listp where b = 1;
-- partitions before finally detecting the correct set of 2nd level partitions
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,1);
QUERY PLAN
----------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3312,7 +3312,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
Filter: (b = ANY (ARRAY[$1, $2]))
(4 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (2,2);
QUERY PLAN
----------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3322,7 +3322,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
(4 rows)
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (0,0);
QUERY PLAN
-----------------------------------
Append (actual rows=0.00 loops=1)
@@ -3333,7 +3333,7 @@ deallocate q1;
-- Test more complex cases where a not-equal condition further eliminates partitions.
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,0);
QUERY PLAN
-------------------------------------------------------------------------
Append (actual rows=0.00 loops=1)
@@ -3343,7 +3343,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
(4 rows)
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,1);
QUERY PLAN
-----------------------------------
Append (actual rows=0.00 loops=1)
@@ -3351,7 +3351,7 @@ explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (
(2 rows)
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select null::int);
QUERY PLAN
------------------------------------------------------
@@ -3376,7 +3376,7 @@ create table stable_qual_pruning2 partition of stable_qual_pruning
create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < localtimestamp;
QUERY PLAN
-----------------------------------------------------------------------------------------
@@ -3389,7 +3389,7 @@ select * from stable_qual_pruning where a < localtimestamp;
(6 rows)
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
QUERY PLAN
-----------------------------------------------------------------------------------------
@@ -3400,7 +3400,7 @@ select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
(4 rows)
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
QUERY PLAN
@@ -3410,7 +3410,7 @@ select * from stable_qual_pruning
One-Time Filter: false
(3 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
QUERY PLAN
@@ -3419,7 +3419,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000","Fri Jan 01 00:00:00 2010"}'::timestamp without time zone[]))
(2 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
QUERY PLAN
@@ -3430,7 +3430,7 @@ select * from stable_qual_pruning
Filter: (a = ANY (ARRAY['Tue Feb 01 00:00:00 2000'::timestamp without time zone, LOCALTIMESTAMP]))
(4 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
QUERY PLAN
@@ -3439,7 +3439,7 @@ select * from stable_qual_pruning
Subplans Removed: 3
(2 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
QUERY PLAN
@@ -3450,7 +3450,7 @@ select * from stable_qual_pruning
Filter: (a = ANY ('{"Tue Feb 01 00:00:00 2000 PST","Fri Jan 01 00:00:00 2010 PST"}'::timestamp with time zone[]))
(4 rows)
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
QUERY PLAN
@@ -3579,7 +3579,7 @@ insert into ma_test select x,x from generate_series(0,29) t(x);
create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(15);
QUERY PLAN
--------------------------------------------------------------------------------------------
Merge Append (actual rows=2.00 loops=1)
@@ -3602,7 +3602,7 @@ execute mt_q1(15);
25
(2 rows)
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(25);
QUERY PLAN
--------------------------------------------------------------------------------------------
Merge Append (actual rows=1.00 loops=1)
@@ -3621,7 +3621,7 @@ execute mt_q1(25);
(1 row)
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(35);
QUERY PLAN
-----------------------------------------
Merge Append (actual rows=0.00 loops=1)
@@ -3637,7 +3637,7 @@ execute mt_q1(35);
deallocate mt_q1;
prepare mt_q2 (int) as select * from ma_test where a >= $1 order by b limit 1;
-- Ensure output list looks sane when the MergeAppend has no subplans.
-explain (analyze, verbose, costs off, summary off, timing off, buffers off) execute mt_q2 (35);
+explain (analyze, verbose, costs off, summary off, timing off, buffers off, io off) execute mt_q2 (35);
QUERY PLAN
-----------------------------------------------
Limit (actual rows=0.00 loops=1)
@@ -3649,7 +3649,7 @@ explain (analyze, verbose, costs off, summary off, timing off, buffers off) exec
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
QUERY PLAN
--------------------------------------------------------------------------------------------------
Merge Append (actual rows=20.00 loops=1)
@@ -4108,7 +4108,7 @@ create table listp (a int, b int) partition by list (a);
create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select 2) and b <> 10;
QUERY PLAN
--------------------------------------------------------
@@ -4235,7 +4235,7 @@ create table rangep_0_to_100_3 partition of rangep_0_to_100 for values in(3);
create table rangep_100_to_200 partition of rangep for values from (100) to (200);
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off, buffers off)
+explain (analyze on, costs off, timing off, summary off, buffers off, io off)
select * from rangep where b IN((select 1),(select 2)) order by a;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index c5968932aed..18826ed4717 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -480,8 +480,8 @@ set enable_indexonlyscan = off;
prepare ab_q1 (int, int, int) as
select * from ab where a between $1 and $2 and b <= $3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2, 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (1, 2, 3);
deallocate ab_q1;
@@ -489,21 +489,21 @@ deallocate ab_q1;
prepare ab_q1 (int, int) as
select a from ab where a between $1 and $2 and b < 3;
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 2);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q1 (2, 4);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q1 (2, 4);
-- Ensure a mix of PARAM_EXTERN and PARAM_EXEC Params work together at
-- different levels of partitioning.
prepare ab_q2 (int, int) as
select a from ab where a between $1 and $2 and b < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q2 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q2 (2, 2);
-- As above, but swap the PARAM_EXEC Param to the first partition level
prepare ab_q3 (int, int) as
select a from ab where b between $1 and $2 and a < (select 3);
-explain (analyze, costs off, summary off, timing off, buffers off) execute ab_q3 (2, 2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute ab_q3 (2, 2);
--
-- Test runtime pruning with hash partitioned tables
@@ -673,11 +673,11 @@ explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from ab where a = (select max(a) from lprt_a) and b = (select max(a)-1 from lprt_a);
-- Test run-time partition pruning with UNION ALL parents
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all select * from ab) ab where b = (select 1);
-- A case containing a UNION ALL with a non-partitioned child.
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from (select * from ab where a = 1 union all (values(10,5)) union all select * from ab) ab where b = (select 1);
-- Another UNION ALL test, but containing a mix of exec init and exec run-time pruning.
@@ -818,7 +818,7 @@ prepare part_abc_q1 (int, int, int) as
select * from part_abc where a = $1 and b = $2 and c = $3;
-- Single partition should be scanned.
-explain (analyze, costs off, summary off, timing off, buffers off) execute part_abc_q1 (1, 2, 3);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute part_abc_q1 (1, 2, 3);
deallocate part_abc_q1;
@@ -838,12 +838,12 @@ select * from listp where b = 1;
-- which match the given parameter.
prepare q1 (int,int) as select * from listp where b in ($1,$2);
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,1);
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (2,2);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (2,2);
-- Try with no matching partitions.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (0,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (0,0);
deallocate q1;
@@ -851,13 +851,13 @@ deallocate q1;
prepare q1 (int,int,int,int) as select * from listp where b in($1,$2) and $3 <> b and $4 <> b;
-- Both partitions allowed by IN clause, but one disallowed by <> clause
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,0);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,0);
-- Both partitions allowed by IN clause, then both excluded again by <> clauses.
-explain (analyze, costs off, summary off, timing off, buffers off) execute q1 (1,2,2,1);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute q1 (1,2,2,1);
-- Ensure Params that evaluate to NULL properly prune away all partitions
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select null::int);
drop table listp;
@@ -874,30 +874,30 @@ create table stable_qual_pruning3 partition of stable_qual_pruning
for values from ('3000-02-01') to ('3000-03-01');
-- comparison against a stable value requires run-time pruning
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < localtimestamp;
-- timestamp < timestamptz comparison is only stable, not immutable
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning where a < '2000-02-01'::timestamptz;
-- check ScalarArrayOp cases
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamp[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', localtimestamp]::timestamp[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2010-02-01', '2020-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(array['2000-02-01', '2010-01-01']::timestamptz[]);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from stable_qual_pruning
where a = any(null::timestamptz[]);
@@ -969,12 +969,12 @@ create index on ma_test (b);
analyze ma_test;
prepare mt_q1 (int) as select a from ma_test where a >= $1 and a % 10 = 5 order by b;
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(15);
execute mt_q1(15);
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(25);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(25);
execute mt_q1(25);
-- Ensure MergeAppend behaves correctly when no subplans match
-explain (analyze, costs off, summary off, timing off, buffers off) execute mt_q1(35);
+explain (analyze, costs off, summary off, timing off, buffers off, io off) execute mt_q1(35);
execute mt_q1(35);
deallocate mt_q1;
@@ -982,12 +982,12 @@ deallocate mt_q1;
prepare mt_q2 (int) as select * from ma_test where a >= $1 order by b limit 1;
-- Ensure output list looks sane when the MergeAppend has no subplans.
-explain (analyze, verbose, costs off, summary off, timing off, buffers off) execute mt_q2 (35);
+explain (analyze, verbose, costs off, summary off, timing off, buffers off, io off) execute mt_q2 (35);
deallocate mt_q2;
-- ensure initplan params properly prune partitions
-explain (analyze, costs off, summary off, timing off, buffers off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
+explain (analyze, costs off, summary off, timing off, buffers off, io off) select * from ma_test where a >= (select min(b) from ma_test_p2) order by b;
reset enable_seqscan;
reset enable_sort;
@@ -1167,7 +1167,7 @@ create table listp1 partition of listp for values in(1);
create table listp2 partition of listp for values in(2) partition by list(b);
create table listp2_10 partition of listp2 for values in (10);
-explain (analyze, costs off, summary off, timing off, buffers off)
+explain (analyze, costs off, summary off, timing off, buffers off, io off)
select * from listp where a = (select 2) and b <> 10;
--
@@ -1235,7 +1235,7 @@ create table rangep_100_to_200 partition of rangep for values from (100) to (200
create index on rangep (a);
-- Ensure run-time pruning works on the nested Merge Append
-explain (analyze on, costs off, timing off, summary off, buffers off)
+explain (analyze on, costs off, timing off, summary off, buffers off, io off)
select * from rangep where b IN((select 1),(select 2)) order by a;
reset enable_sort;
drop table rangep;
--
2.43.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-05 22:46 ` Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-05 22:46 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/6/26 00:18, Melanie Plageman wrote:
> On Fri, Apr 3, 2026 at 3:01 PM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> 1) stats for non-parallel-aware scans
>>
>> It correctly shows stats for scans that are not parallel-aware.
> <--snip-->
>> For SeqScan/TidRangeScan I've merged the changes into 0004 and 0005.
>>
>> I have to admit I'm not a huge fan of the changes required to fix this.
>> The logic with all these instument/parallel_aware conditions seems much
>> less readable to me, but I don't know how else to do this. The approach
>> is copied from nodeIndexscan.c, which already did that in 0fbceae841cb.
>
> While this is the opposite direction of what I suggested to fix BHS in
> [1], what if we allocated the instrumentation and parallel-aware state
> separately and accessed them with their own keys? It's a little janky
> because what key could we use besides the plan_node_id, but if we add
> a key-sized offset to the plan node id, we can functionally have two
> separate keys.
>
Interesting idea! That'd mean we don't need to mess with the
instrument/parallel_aware flags in the functions. That seems like an
improvement, it made the code ... not nice.
Presumably, we'd only do this in master? It seems way too invasive to
backpatch (and for index scans it'd even be an ABI break, so we can't do
that). Moreover, for index scans it's not even a bug, and it does not
seem great to do it one way for BHS and a completely different way in
index scans.
So we'd either not fix BHS in backbranches, or do it the "ugly" way.
> We could do this for all nodes that have both shared instrumentation
> and parallel-aware shared state. It introduces some boilerplate
> functions, but I do find it easier to understand.
>
> I've attached patches that implement this to make the idea more clear.
> 0002 does the two allocations for the existing index[-only] scan
> nodes. 0003 uses this method to fix BHS. 0004 is your patch to add
> prefetch stuff to explain and show it for BHS. And 0005 and 0006 are
> using the two allocation approach for seqscan and tidrange scan. There
> were some test changes you had that didn't seem required to make tests
> pass in the commits that I stuffed into 0007. I used an LLM to do some
> of the boilerplate and rebasing, so there might be some weirdness in
> there.
Thanks! It's a bit late for me to experiment with this, but I'll take a
closer look tomorrow. We're getting pretty clear to feature freeze, and
these seem like big changes ...
>
> If we don't do the above, then I think your current approach is the
> only other realistic option. We can't do what I suggested for BHS in
> [1] and always allocate the parallel-aware state because that state is
> much larger for sequential scans and TID range scans.
>
Yeah. I was wondering about these costs when you proposed to allocate
the BHS parallel state always. I concluded it does not matter for BHS,
but for scans with larger states it might be different.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-05 22:50 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-05 22:50 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Sun, Apr 5, 2026 at 6:46 PM Tomas Vondra <tomas@vondra.me> wrote:
>
> On 4/6/26 00:18, Melanie Plageman wrote:
>
> > While this is the opposite direction of what I suggested to fix BHS in
> > [1], what if we allocated the instrumentation and parallel-aware state
> > separately and accessed them with their own keys? It's a little janky
> > because what key could we use besides the plan_node_id, but if we add
> > a key-sized offset to the plan node id, we can functionally have two
> > separate keys.
>
> Presumably, we'd only do this in master? It seems way too invasive to
> backpatch (and for index scans it'd even be an ABI break, so we can't do
> that). Moreover, for index scans it's not even a bug, and it does not
> seem great to do it one way for BHS and a completely different way in
> index scans.
>
> So we'd either not fix BHS in backbranches, or do it the "ugly" way.
I think we could backpatch the BHS patch I posted in the other thread
that always allocated pstate. It's a very small diff and seems quite
low risk to me. Though that struct changed a lot in 17 I think
(courtesy of me), so backpatching might be a little bit of a headache
in earlier versions.
> > If we don't do the above, then I think your current approach is the
> > only other realistic option. We can't do what I suggested for BHS in
> > [1] and always allocate the parallel-aware state because that state is
> > much larger for sequential scans and TID range scans.
>
> Yeah. I was wondering about these costs when you proposed to allocate
> the BHS parallel state always. I concluded it does not matter for BHS,
> but for scans with larger states it might be different.
Yea, I think ParallelBitmapHeapState is like 48 bytes or something
- Melanie
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-06 09:14 ` Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-06 09:14 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/6/26 00:50, Melanie Plageman wrote:
> On Sun, Apr 5, 2026 at 6:46 PM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> On 4/6/26 00:18, Melanie Plageman wrote:
>>
>>> While this is the opposite direction of what I suggested to fix BHS in
>>> [1], what if we allocated the instrumentation and parallel-aware state
>>> separately and accessed them with their own keys? It's a little janky
>>> because what key could we use besides the plan_node_id, but if we add
>>> a key-sized offset to the plan node id, we can functionally have two
>>> separate keys.
>>
>> Presumably, we'd only do this in master? It seems way too invasive to
>> backpatch (and for index scans it'd even be an ABI break, so we can't do
>> that). Moreover, for index scans it's not even a bug, and it does not
>> seem great to do it one way for BHS and a completely different way in
>> index scans.
>>
>> So we'd either not fix BHS in backbranches, or do it the "ugly" way.
>
> I think we could backpatch the BHS patch I posted in the other thread
> that always allocated pstate. It's a very small diff and seems quite
> low risk to me. Though that struct changed a lot in 17 I think
> (courtesy of me), so backpatching might be a little bit of a headache
> in earlier versions.
>
Makes sense.
>>> If we don't do the above, then I think your current approach is the
>>> only other realistic option. We can't do what I suggested for BHS in
>>> [1] and always allocate the parallel-aware state because that state is
>>> much larger for sequential scans and TID range scans.
>>
>> Yeah. I was wondering about these costs when you proposed to allocate
>> the BHS parallel state always. I concluded it does not matter for BHS,
>> but for scans with larger states it might be different.
>
> Yea, I think ParallelBitmapHeapState is like 48 bytes or something
>
Right. TBH after thinking about it I can't imagine this being a serious
issue even for larger states, considering how expensive it is to setup
parallel workers etc. That'd just dwarf this cost, I think.
Anyway, that doesn't matter much, because the more I look at the
approach with having a separate chunk of shared memory, the more I like
it. It seems much simpler, more elegant, etc. I really disliked how
unreadable the code got with the parallel_aware/instrument checks in
multiple places, and this just gets rid of that. I like that.
I was worried it might be adding a non-trivial amount of overhead, but
after refreshing my memory of how the shm works, I think there's no
change at all.
Is execParallel.h the right place to define the offset? It means the
various nodes (like nodeBitmapHeapScan) now have to include this header,
and it seems a bit suspicious. I can't think of a better .h file, and
maybe I'm wrong and it's perfectly fine.
Regarding plan_node_id - I think the offset works fine for now. It
effectively gives us 2 IDs per node. The only alternative I can think of
is having nodes "request" how many IDs will be needed - most nodes would
say "1", nodes with instrumentation would say "2", etc. In the future we
might get a node that needs 3+ shm chunks. I don't think we need to do
all that now.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-06 14:01 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-06 14:01 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 6, 2026 at 5:15 AM Tomas Vondra <tomas@vondra.me> wrote:
>
> Anyway, that doesn't matter much, because the more I look at the
> approach with having a separate chunk of shared memory, the more I like
> it. It seems much simpler, more elegant, etc. I really disliked how
> unreadable the code got with the parallel_aware/instrument checks in
> multiple places, and this just gets rid of that. I like that.
>
<--snip-->
>
> Is execParallel.h the right place to define the offset? It means the
> various nodes (like nodeBitmapHeapScan) now have to include this header,
> and it seems a bit suspicious. I can't think of a better .h file, and
> maybe I'm wrong and it's perfectly fine.
It could go in src/include/executor is instrument_node.h. It's where
the structs for the shared instrumentation go. It's probably cheaper
to include also because it doesn't include anything.
> Regarding plan_node_id - I think the offset works fine for now. It
> effectively gives us 2 IDs per node. The only alternative I can think of
> is having nodes "request" how many IDs will be needed - most nodes would
> say "1", nodes with instrumentation would say "2", etc. In the future we
> might get a node that needs 3+ shm chunks. I don't think we need to do
> all that now.
Makes sense to me.
- Melanie
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-06 16:50 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:33 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 2 replies; 59+ messages in thread
From: Melanie Plageman @ 2026-04-06 16:50 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 6, 2026 at 10:01 AM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> On Mon, Apr 6, 2026 at 5:15 AM Tomas Vondra <tomas@vondra.me> wrote:
> >
> > Anyway, that doesn't matter much, because the more I look at the
> > approach with having a separate chunk of shared memory, the more I like
> > it. It seems much simpler, more elegant, etc. I really disliked how
> > unreadable the code got with the parallel_aware/instrument checks in
> > multiple places, and this just gets rid of that. I like that.
> >
> <--snip-->
> >
> > Is execParallel.h the right place to define the offset? It means the
> > various nodes (like nodeBitmapHeapScan) now have to include this header,
> > and it seems a bit suspicious. I can't think of a better .h file, and
> > maybe I'm wrong and it's perfectly fine.
>
> It could go in src/include/executor is instrument_node.h. It's where
> the structs for the shared instrumentation go. It's probably cheaper
> to include also because it doesn't include anything.
I cleaned up the first patch in the set that refactors index-only and
index scan to use this pattern and realized that I wasn't sure what to
do about the duplication between index-only and index scans for these
functions.
ExecIndexScanInstrumentEstimate() and
ExecIndexOnlyScanInstrumentEstimate() are the same code except they
take different node types to do the check for intsrumentation
if (!node->ss.ps.instrument || pcxt->nworkers == 0)
return;
The size estimation for SharedIndexScanIntrumentation could be common
to the two but it is only three lines and doesn't seem worth another
function call. ExecIndex[Only]ScanEstimate() use a common helper
index_parallelscan_estimate() but that is more code.
We could just have a common helper and only call it if
node->ss.ps.instrument && pcxt->nworkers > 0 but I don't see anybody
else in ExecParallelEstimate() checking for instrumentation being
enabled.
Besides that, the function names are very long already but don't
include the word parallel (though ExecIndexScanEstimate() is for
parallel scans only and doesn't include "parallel").
ExecIndex[Only]ScanInstrumentInitDSM() is the same between the two
except the node type it takes but three different members of node are
used here, so I'm not sure how much sense a common helper makes here.
I have a feeling the above stuff isn't okay how it is now, but I'm not
sure which direction to go.
I think ExecIndex[Only]ScanInstrumentInitWorker() is fine the way it
is because it is only 1 line and requires a different node for
index/index-only.
- Melanie
Attachments:
[text/x-patch] v9-0001-Allocate-separate-DSM-chunk-for-parallel-Index-On.patch (23.1K, ../../CAAKRu_ZQ_yapZoMWt-WuUfwcZyw4hek81Oo7x2_yTqLodzCjCQ@mail.gmail.com/2-v9-0001-Allocate-separate-DSM-chunk-for-parallel-Index-On.patch)
download | inline diff:
From f68de64b8747c9fe1e71fbcbd2285b7f1fc3a41a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 6 Apr 2026 12:26:15 -0400
Subject: [PATCH v9] Allocate separate DSM chunk for parallel Index[Only]Scan
instrumentation
Parallel index and index-only scans may or may not be parallel-aware and
may or may not be instrumented (for EXPLAIN ANALYZE). Non-parallel-aware
scans still need shared instrumentation to report per-worker statistics
to the leader. Because all combinations of parallel-aware and
instrumented are possible, it is cleaner to allocate separate DSM chunks
for the shared instrumentation and the parallel scan descriptor. This is
of particular interest now because a future commit will extend this
pattern to other scan node types.
---
src/backend/access/index/indexam.c | 51 ++--------
src/backend/executor/execParallel.c | 40 ++++++--
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeIndexonlyscan.c | 108 ++++++++++++--------
src/backend/executor/nodeIndexscan.c | 113 +++++++++++++--------
src/include/access/genam.h | 7 +-
src/include/access/relscan.h | 1 -
src/include/executor/instrument_node.h | 11 ++
src/include/executor/nodeIndexonlyscan.h | 6 ++
src/include/executor/nodeIndexscan.h | 6 ++
10 files changed, 198 insertions(+), 154 deletions(-)
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 23288a4f994..7967e939847 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -463,43 +463,26 @@ index_restrpos(IndexScanDesc scan)
}
/*
- * index_parallelscan_estimate - estimate shared memory for parallel scan
- *
- * When instrument=true, estimate includes SharedIndexScanInstrumentation
- * space. When parallel_aware=true, estimate includes whatever space the
- * index AM's amestimateparallelscan routine requested when called.
+ * Estimates the shared memory needed for parallel scan, including any
+ * AM-specific parallel scan state.
*/
Size
index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
- Snapshot snapshot, bool instrument,
- bool parallel_aware, int nworkers)
+ Snapshot snapshot)
{
Size nbytes;
- Assert(instrument || parallel_aware);
-
RELATION_CHECKS;
nbytes = offsetof(ParallelIndexScanDescData, ps_snapshot_data);
nbytes = add_size(nbytes, EstimateSnapshotSpace(snapshot));
nbytes = MAXALIGN(nbytes);
- if (instrument)
- {
- Size sharedinfosz;
-
- sharedinfosz = offsetof(SharedIndexScanInstrumentation, winstrument) +
- nworkers * sizeof(IndexScanInstrumentation);
- nbytes = add_size(nbytes, sharedinfosz);
- nbytes = MAXALIGN(nbytes);
- }
-
/*
* If parallel scan index AM interface can't be used (or index AM provides
* no such interface), assume there is no AM-specific data needed
*/
- if (parallel_aware &&
- indexRelation->rd_indam->amestimateparallelscan != NULL)
+ if (indexRelation->rd_indam->amestimateparallelscan != NULL)
nbytes = add_size(nbytes,
indexRelation->rd_indam->amestimateparallelscan(indexRelation,
nkeys,
@@ -520,15 +503,11 @@ index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
*/
void
index_parallelscan_initialize(Relation heapRelation, Relation indexRelation,
- Snapshot snapshot, bool instrument,
- bool parallel_aware, int nworkers,
- SharedIndexScanInstrumentation **sharedinfo,
+ Snapshot snapshot,
ParallelIndexScanDesc target)
{
Size offset;
- Assert(instrument || parallel_aware);
-
RELATION_CHECKS;
offset = add_size(offsetof(ParallelIndexScanDescData, ps_snapshot_data),
@@ -537,29 +516,11 @@ index_parallelscan_initialize(Relation heapRelation, Relation indexRelation,
target->ps_locator = heapRelation->rd_locator;
target->ps_indexlocator = indexRelation->rd_locator;
- target->ps_offset_ins = 0;
target->ps_offset_am = 0;
SerializeSnapshot(snapshot, target->ps_snapshot_data);
- if (instrument)
- {
- Size sharedinfosz;
-
- target->ps_offset_ins = offset;
- sharedinfosz = offsetof(SharedIndexScanInstrumentation, winstrument) +
- nworkers * sizeof(IndexScanInstrumentation);
- offset = add_size(offset, sharedinfosz);
- offset = MAXALIGN(offset);
-
- /* Set leader's *sharedinfo pointer, and initialize stats */
- *sharedinfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(target, target->ps_offset_ins);
- memset(*sharedinfo, 0, sharedinfosz);
- (*sharedinfo)->num_workers = nworkers;
- }
-
/* aminitparallelscan is optional; assume no-op if not provided by AM */
- if (parallel_aware && indexRelation->rd_indam->aminitparallelscan != NULL)
+ if (indexRelation->rd_indam->aminitparallelscan != NULL)
{
void *amtarget;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 5e4a4a9740c..726aca230a4 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -259,14 +259,20 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
e->pcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanEstimate((IndexScanState *) planstate,
+ e->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanEstimate((IndexScanState *) planstate,
- e->pcxt);
+ ExecIndexScanInstrumentEstimate((IndexScanState *) planstate,
+ e->pcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
- e->pcxt);
+ ExecIndexOnlyScanInstrumentEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -493,13 +499,20 @@ ExecParallelInitializeDSM(PlanState *planstate,
d->pcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanInitializeDSM((IndexScanState *) planstate,
+ d->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanInitializeDSM((IndexScanState *) planstate, d->pcxt);
+ ExecIndexScanInstrumentInitDSM((IndexScanState *) planstate,
+ d->pcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
- d->pcxt);
+ ExecIndexOnlyScanInstrumentInitDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -1371,13 +1384,20 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanInitializeWorker((IndexScanState *) planstate,
+ pwcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanInitializeWorker((IndexScanState *) planstate, pwcxt);
+ ExecIndexScanInstrumentInitWorker((IndexScanState *) planstate,
+ pwcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate,
+ pwcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate,
- pwcxt);
+ ExecIndexOnlyScanInstrumentInitWorker((IndexOnlyScanState *) planstate,
+ pwcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 70c55ee6d61..7978514e1bc 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -394,7 +394,9 @@ ExecBitmapIndexScanInitializeDSM(BitmapIndexScanState *node,
node->biss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc,
size);
- shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id,
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
node->biss_SharedInfo);
/* Each per-worker area must start out as zeroes */
@@ -417,7 +419,10 @@ ExecBitmapIndexScanInitializeWorker(BitmapIndexScanState *node,
return;
node->biss_SharedInfo = (SharedIndexScanInstrumentation *)
- shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index de6154fd541..3e4cc0b9402 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -736,21 +736,11 @@ ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
- bool instrument = (node->ss.ps.instrument != NULL);
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- estate->es_snapshot,
- instrument, parallel_aware,
- pcxt->nworkers);
+ estate->es_snapshot);
shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -767,29 +757,14 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
index_parallelscan_initialize(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
estate->es_snapshot,
- instrument, parallel_aware, pcxt->nworkers,
- &node->ioss_SharedInfo, piscan);
+ piscan);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
- if (!parallel_aware)
- {
- /* Only here to initialize SharedInfo in DSM */
- return;
- }
-
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
@@ -837,27 +812,9 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- if (instrument)
- node->ioss_SharedInfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(piscan, piscan->ps_offset_ins);
-
- if (!parallel_aware)
- {
- /* Only here to set up worker node's SharedInfo */
- return;
- }
-
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
@@ -879,6 +836,67 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
}
+/*
+ * Compute the amount of DSM we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel index-only scan instrumentation.
+ */
+void
+ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ node->ioss_SharedInfo =
+ (SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->ioss_SharedInfo, 0, size);
+ node->ioss_SharedInfo->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->ioss_SharedInfo);
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecIndexOnlyScanInstrumentInitWorker(IndexOnlyScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->ioss_SharedInfo = (SharedIndexScanInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
/* ----------------------------------------------------------------
* ExecIndexOnlyScanRetrieveInstrumentation
*
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 1620d146071..095ce538f04 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -1674,21 +1674,11 @@ ExecIndexScanEstimate(IndexScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
node->iss_PscanLen = index_parallelscan_estimate(node->iss_RelationDesc,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- estate->es_snapshot,
- instrument, parallel_aware,
- pcxt->nworkers);
+ estate->es_snapshot);
shm_toc_estimate_chunk(&pcxt->estimator, node->iss_PscanLen);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -1705,29 +1695,14 @@ ExecIndexScanInitializeDSM(IndexScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_allocate(pcxt->toc, node->iss_PscanLen);
index_parallelscan_initialize(node->ss.ss_currentRelation,
node->iss_RelationDesc,
estate->es_snapshot,
- instrument, parallel_aware, pcxt->nworkers,
- &node->iss_SharedInfo, piscan);
+ piscan);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
- if (!parallel_aware)
- {
- /* Only here to initialize SharedInfo in DSM */
- return;
- }
-
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
@@ -1773,27 +1748,9 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- if (instrument)
- node->iss_SharedInfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(piscan, piscan->ps_offset_ins);
-
- if (!parallel_aware)
- {
- /* Only here to set up worker node's SharedInfo */
- return;
- }
-
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
@@ -1814,6 +1771,72 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
node->iss_OrderByKeys, node->iss_NumOrderByKeys);
}
+/*
+ * Compute the amount of DSM we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecIndexScanInstrumentEstimate(IndexScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ /*
+ * This size calculation is trivial enough that we don't bother saving it
+ * in the IndexScanState. We'll recalculate the needed size in
+ * ExecIndexScanInstrumentInitDSM().
+ */
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel index scan instrumentation.
+ */
+void
+ExecIndexScanInstrumentInitDSM(IndexScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ /* don't need this if not instrumenting or no workers */
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ node->iss_SharedInfo =
+ (SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->iss_SharedInfo, 0, size);
+ node->iss_SharedInfo->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->iss_SharedInfo);
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecIndexScanInstrumentInitWorker(IndexScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->iss_SharedInfo = (SharedIndexScanInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
/* ----------------------------------------------------------------
* ExecIndexScanRetrieveInstrumentation
*
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index b69320a7fc8..68bfe405db3 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -171,14 +171,9 @@ extern void index_endscan(IndexScanDesc scan);
extern void index_markpos(IndexScanDesc scan);
extern void index_restrpos(IndexScanDesc scan);
extern Size index_parallelscan_estimate(Relation indexRelation,
- int nkeys, int norderbys, Snapshot snapshot,
- bool instrument, bool parallel_aware,
- int nworkers);
+ int nkeys, int norderbys, Snapshot snapshot);
extern void index_parallelscan_initialize(Relation heapRelation,
Relation indexRelation, Snapshot snapshot,
- bool instrument, bool parallel_aware,
- int nworkers,
- SharedIndexScanInstrumentation **sharedinfo,
ParallelIndexScanDesc target);
extern void index_parallelrescan(IndexScanDesc scan);
extern IndexScanDesc index_beginscan_parallel(Relation heaprel,
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 960abf6c214..fd2076c582a 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -203,7 +203,6 @@ typedef struct ParallelIndexScanDescData
{
RelFileLocator ps_locator; /* physical table relation to scan */
RelFileLocator ps_indexlocator; /* physical index relation to scan */
- Size ps_offset_ins; /* Offset to SharedIndexScanInstrumentation */
Size ps_offset_am; /* Offset to am-specific structure */
char ps_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
} ParallelIndexScanDescData;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 2a0ff377a73..46db2dd70c4 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -18,6 +18,17 @@
#ifndef INSTRUMENT_NODE_H
#define INSTRUMENT_NODE_H
+/*
+ * Offset added to plan_node_id to create a second TOC key for per-worker scan
+ * instrumentation. This allows scan nodes to store instrumentation in a
+ * separate DSM allocation from their parallel scan descriptor. This is useful
+ * because scans may be non-parallel-aware but still require instrumentation or
+ * parallel-aware and not require instrumentation. Having two allocations
+ * provides flexibility to allocate and initialize what is needed. In the
+ * future, if nodes need more than two DSM allocations, we would need a more
+ * robust system.
+ */
+#define PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET UINT64CONST(0xD000000000000000)
/* ---------------------
* Instrumentation information for aggregate function execution
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index ba807b8d8d9..b686244eb91 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -32,6 +32,12 @@ extern void ExecIndexOnlyScanReInitializeDSM(IndexOnlyScanState *node,
ParallelContext *pcxt);
extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInstrumentInitWorker(IndexOnlyScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecIndexOnlyScanRetrieveInstrumentation(IndexOnlyScanState *node);
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/executor/nodeIndexscan.h b/src/include/executor/nodeIndexscan.h
index 803be5b08df..c60cbc9a94b 100644
--- a/src/include/executor/nodeIndexscan.h
+++ b/src/include/executor/nodeIndexscan.h
@@ -28,6 +28,12 @@ extern void ExecIndexScanInitializeDSM(IndexScanState *node, ParallelContext *pc
extern void ExecIndexScanReInitializeDSM(IndexScanState *node, ParallelContext *pcxt);
extern void ExecIndexScanInitializeWorker(IndexScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecIndexScanInstrumentEstimate(IndexScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexScanInstrumentInitDSM(IndexScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexScanInstrumentInitWorker(IndexScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecIndexScanRetrieveInstrumentation(IndexScanState *node);
/*
--
2.43.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-06 20:33 ` Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 0 replies; 59+ messages in thread
From: Melanie Plageman @ 2026-04-06 20:33 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 6, 2026 at 12:50 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> I cleaned up the first patch in the set that refactors index-only and
> index scan to use this pattern and realized that I wasn't sure what to
> do about the duplication between index-only and index scans for these
> functions.
I changed my mind about the duplication: I think it's fine because
there just aren't that many lines of code. Attached v10 has a
preparatory commit that adds an assert-only check to shm_toc_insert()
to make sure the key hasn't already been inserted. I thought it would
be helpful since many of the magic numbers for parallel executor
communication are spread around.
- Melanie
Attachments:
[text/x-patch] v10-0001-Assert-no-duplicate-keys-in-shm_toc_insert.patch (1.1K, ../../CAAKRu_b1R+syKRvKc+wD1H7jX8QUFWcWQw7HtC48XeGdBRuU0w@mail.gmail.com/2-v10-0001-Assert-no-duplicate-keys-in-shm_toc_insert.patch)
download | inline diff:
From 671ac63b3f443b22b4e268c7f6d1f48ee15e99df Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 6 Apr 2026 15:16:05 -0400
Subject: [PATCH v10 1/2] Assert no duplicate keys in shm_toc_insert()
shm_toc_insert() accepts duplicate keys, and a duplicate would go
undetected. Add an assertion to catch this. Keys are defined throughout
the executor, so it is useful of having a way to detect collisions.
---
src/backend/storage/ipc/shm_toc.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/src/backend/storage/ipc/shm_toc.c b/src/backend/storage/ipc/shm_toc.c
index e74a0b97ac0..f3bbb24bfdd 100644
--- a/src/backend/storage/ipc/shm_toc.c
+++ b/src/backend/storage/ipc/shm_toc.c
@@ -201,6 +201,13 @@ shm_toc_insert(shm_toc *toc, uint64 key, void *address)
}
Assert(offset < total_bytes);
+
+#ifdef USE_ASSERT_CHECKING
+ /* Verify no duplicate keys */
+ for (Size i = 0; i < nentry; i++)
+ Assert(vtoc->toc_entry[i].key != key);
+#endif
+
vtoc->toc_entry[nentry].key = key;
vtoc->toc_entry[nentry].offset = offset;
--
2.43.0
[text/x-patch] v10-0002-Allocate-separate-DSM-chunk-for-parallel-Index-O.patch (22.9K, ../../CAAKRu_b1R+syKRvKc+wD1H7jX8QUFWcWQw7HtC48XeGdBRuU0w@mail.gmail.com/3-v10-0002-Allocate-separate-DSM-chunk-for-parallel-Index-O.patch)
download | inline diff:
From fc3f5fe6521fb4fef10d214b4c2efdb9f8fe5733 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 6 Apr 2026 12:26:15 -0400
Subject: [PATCH v10 2/2] Allocate separate DSM chunk for parallel
Index[Only]Scan instrumentation
Previously, parallel index and index-only scans packed the parallel scan
descriptor and shared instrumentation (for EXPLAIN ANALYZE) into a
single DSM allocation. Since scans may be instrumented without being
parallel-aware, and vice versa, using separate DSM chunks -- each with
its own TOC key -- is cleaner. A future commit will extend this pattern
to other scan node types.
---
src/backend/access/index/indexam.c | 51 ++--------
src/backend/executor/execParallel.c | 40 ++++++--
src/backend/executor/nodeBitmapIndexscan.c | 9 +-
src/backend/executor/nodeIndexonlyscan.c | 111 ++++++++++++---------
src/backend/executor/nodeIndexscan.c | 111 ++++++++++++---------
src/include/access/genam.h | 7 +-
src/include/access/relscan.h | 1 -
src/include/executor/instrument_node.h | 11 ++
src/include/executor/nodeIndexonlyscan.h | 6 ++
src/include/executor/nodeIndexscan.h | 6 ++
10 files changed, 199 insertions(+), 154 deletions(-)
diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c
index 23288a4f994..7967e939847 100644
--- a/src/backend/access/index/indexam.c
+++ b/src/backend/access/index/indexam.c
@@ -463,43 +463,26 @@ index_restrpos(IndexScanDesc scan)
}
/*
- * index_parallelscan_estimate - estimate shared memory for parallel scan
- *
- * When instrument=true, estimate includes SharedIndexScanInstrumentation
- * space. When parallel_aware=true, estimate includes whatever space the
- * index AM's amestimateparallelscan routine requested when called.
+ * Estimates the shared memory needed for parallel scan, including any
+ * AM-specific parallel scan state.
*/
Size
index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
- Snapshot snapshot, bool instrument,
- bool parallel_aware, int nworkers)
+ Snapshot snapshot)
{
Size nbytes;
- Assert(instrument || parallel_aware);
-
RELATION_CHECKS;
nbytes = offsetof(ParallelIndexScanDescData, ps_snapshot_data);
nbytes = add_size(nbytes, EstimateSnapshotSpace(snapshot));
nbytes = MAXALIGN(nbytes);
- if (instrument)
- {
- Size sharedinfosz;
-
- sharedinfosz = offsetof(SharedIndexScanInstrumentation, winstrument) +
- nworkers * sizeof(IndexScanInstrumentation);
- nbytes = add_size(nbytes, sharedinfosz);
- nbytes = MAXALIGN(nbytes);
- }
-
/*
* If parallel scan index AM interface can't be used (or index AM provides
* no such interface), assume there is no AM-specific data needed
*/
- if (parallel_aware &&
- indexRelation->rd_indam->amestimateparallelscan != NULL)
+ if (indexRelation->rd_indam->amestimateparallelscan != NULL)
nbytes = add_size(nbytes,
indexRelation->rd_indam->amestimateparallelscan(indexRelation,
nkeys,
@@ -520,15 +503,11 @@ index_parallelscan_estimate(Relation indexRelation, int nkeys, int norderbys,
*/
void
index_parallelscan_initialize(Relation heapRelation, Relation indexRelation,
- Snapshot snapshot, bool instrument,
- bool parallel_aware, int nworkers,
- SharedIndexScanInstrumentation **sharedinfo,
+ Snapshot snapshot,
ParallelIndexScanDesc target)
{
Size offset;
- Assert(instrument || parallel_aware);
-
RELATION_CHECKS;
offset = add_size(offsetof(ParallelIndexScanDescData, ps_snapshot_data),
@@ -537,29 +516,11 @@ index_parallelscan_initialize(Relation heapRelation, Relation indexRelation,
target->ps_locator = heapRelation->rd_locator;
target->ps_indexlocator = indexRelation->rd_locator;
- target->ps_offset_ins = 0;
target->ps_offset_am = 0;
SerializeSnapshot(snapshot, target->ps_snapshot_data);
- if (instrument)
- {
- Size sharedinfosz;
-
- target->ps_offset_ins = offset;
- sharedinfosz = offsetof(SharedIndexScanInstrumentation, winstrument) +
- nworkers * sizeof(IndexScanInstrumentation);
- offset = add_size(offset, sharedinfosz);
- offset = MAXALIGN(offset);
-
- /* Set leader's *sharedinfo pointer, and initialize stats */
- *sharedinfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(target, target->ps_offset_ins);
- memset(*sharedinfo, 0, sharedinfosz);
- (*sharedinfo)->num_workers = nworkers;
- }
-
/* aminitparallelscan is optional; assume no-op if not provided by AM */
- if (parallel_aware && indexRelation->rd_indam->aminitparallelscan != NULL)
+ if (indexRelation->rd_indam->aminitparallelscan != NULL)
{
void *amtarget;
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 5e4a4a9740c..726aca230a4 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -259,14 +259,20 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
e->pcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanEstimate((IndexScanState *) planstate,
+ e->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanEstimate((IndexScanState *) planstate,
- e->pcxt);
+ ExecIndexScanInstrumentEstimate((IndexScanState *) planstate,
+ e->pcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanEstimate((IndexOnlyScanState *) planstate,
- e->pcxt);
+ ExecIndexOnlyScanInstrumentEstimate((IndexOnlyScanState *) planstate,
+ e->pcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -493,13 +499,20 @@ ExecParallelInitializeDSM(PlanState *planstate,
d->pcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanInitializeDSM((IndexScanState *) planstate,
+ d->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanInitializeDSM((IndexScanState *) planstate, d->pcxt);
+ ExecIndexScanInstrumentInitDSM((IndexScanState *) planstate,
+ d->pcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanInitializeDSM((IndexOnlyScanState *) planstate,
- d->pcxt);
+ ExecIndexOnlyScanInstrumentInitDSM((IndexOnlyScanState *) planstate,
+ d->pcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
@@ -1371,13 +1384,20 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexScanInitializeWorker((IndexScanState *) planstate,
+ pwcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexScanInitializeWorker((IndexScanState *) planstate, pwcxt);
+ ExecIndexScanInstrumentInitWorker((IndexScanState *) planstate,
+ pwcxt);
break;
case T_IndexOnlyScanState:
+ if (planstate->plan->parallel_aware)
+ ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate,
+ pwcxt);
/* even when not parallel-aware, for EXPLAIN ANALYZE */
- ExecIndexOnlyScanInitializeWorker((IndexOnlyScanState *) planstate,
- pwcxt);
+ ExecIndexOnlyScanInstrumentInitWorker((IndexOnlyScanState *) planstate,
+ pwcxt);
break;
case T_BitmapIndexScanState:
/* even when not parallel-aware, for EXPLAIN ANALYZE */
diff --git a/src/backend/executor/nodeBitmapIndexscan.c b/src/backend/executor/nodeBitmapIndexscan.c
index 70c55ee6d61..7978514e1bc 100644
--- a/src/backend/executor/nodeBitmapIndexscan.c
+++ b/src/backend/executor/nodeBitmapIndexscan.c
@@ -394,7 +394,9 @@ ExecBitmapIndexScanInitializeDSM(BitmapIndexScanState *node,
node->biss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc,
size);
- shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id,
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
node->biss_SharedInfo);
/* Each per-worker area must start out as zeroes */
@@ -417,7 +419,10 @@ ExecBitmapIndexScanInitializeWorker(BitmapIndexScanState *node,
return;
node->biss_SharedInfo = (SharedIndexScanInstrumentation *)
- shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index de6154fd541..d9ea735c2d2 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -736,21 +736,11 @@ ExecIndexOnlyScanEstimate(IndexOnlyScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
- bool instrument = (node->ss.ps.instrument != NULL);
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
node->ioss_PscanLen = index_parallelscan_estimate(node->ioss_RelationDesc,
node->ioss_NumScanKeys,
node->ioss_NumOrderByKeys,
- estate->es_snapshot,
- instrument, parallel_aware,
- pcxt->nworkers);
+ estate->es_snapshot);
shm_toc_estimate_chunk(&pcxt->estimator, node->ioss_PscanLen);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -767,29 +757,14 @@ ExecIndexOnlyScanInitializeDSM(IndexOnlyScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_allocate(pcxt->toc, node->ioss_PscanLen);
index_parallelscan_initialize(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
estate->es_snapshot,
- instrument, parallel_aware, pcxt->nworkers,
- &node->ioss_SharedInfo, piscan);
+ piscan);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
- if (!parallel_aware)
- {
- /* Only here to initialize SharedInfo in DSM */
- return;
- }
-
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
@@ -837,27 +812,9 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- if (instrument)
- node->ioss_SharedInfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(piscan, piscan->ps_offset_ins);
-
- if (!parallel_aware)
- {
- /* Only here to set up worker node's SharedInfo */
- return;
- }
-
node->ioss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->ioss_RelationDesc,
@@ -879,6 +836,70 @@ ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
node->ioss_OrderByKeys, node->ioss_NumOrderByKeys);
}
+/*
+ * Compute the amount of DSM we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ /*
+ * This size calculation is trivial enough that we don't bother saving it
+ * in the IndexOnlyScanState. We'll recalculate the needed size in
+ * ExecIndexOnlyScanInstrumentInitDSM().
+ */
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel index-only scan instrumentation.
+ */
+void
+ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ node->ioss_SharedInfo =
+ (SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->ioss_SharedInfo, 0, size);
+ node->ioss_SharedInfo->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->ioss_SharedInfo);
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecIndexOnlyScanInstrumentInitWorker(IndexOnlyScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->ioss_SharedInfo = (SharedIndexScanInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
/* ----------------------------------------------------------------
* ExecIndexOnlyScanRetrieveInstrumentation
*
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 1620d146071..1ecf22fce7b 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -1674,21 +1674,11 @@ ExecIndexScanEstimate(IndexScanState *node,
ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
node->iss_PscanLen = index_parallelscan_estimate(node->iss_RelationDesc,
node->iss_NumScanKeys,
node->iss_NumOrderByKeys,
- estate->es_snapshot,
- instrument, parallel_aware,
- pcxt->nworkers);
+ estate->es_snapshot);
shm_toc_estimate_chunk(&pcxt->estimator, node->iss_PscanLen);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -1705,29 +1695,14 @@ ExecIndexScanInitializeDSM(IndexScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_allocate(pcxt->toc, node->iss_PscanLen);
index_parallelscan_initialize(node->ss.ss_currentRelation,
node->iss_RelationDesc,
estate->es_snapshot,
- instrument, parallel_aware, pcxt->nworkers,
- &node->iss_SharedInfo, piscan);
+ piscan);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, piscan);
- if (!parallel_aware)
- {
- /* Only here to initialize SharedInfo in DSM */
- return;
- }
-
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
@@ -1773,27 +1748,9 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelIndexScanDesc piscan;
- bool instrument = node->ss.ps.instrument != NULL;
- bool parallel_aware = node->ss.ps.plan->parallel_aware;
-
- if (!instrument && !parallel_aware)
- {
- /* No DSM required by the scan */
- return;
- }
piscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
- if (instrument)
- node->iss_SharedInfo = (SharedIndexScanInstrumentation *)
- OffsetToPointer(piscan, piscan->ps_offset_ins);
-
- if (!parallel_aware)
- {
- /* Only here to set up worker node's SharedInfo */
- return;
- }
-
node->iss_ScanDesc =
index_beginscan_parallel(node->ss.ss_currentRelation,
node->iss_RelationDesc,
@@ -1814,6 +1771,70 @@ ExecIndexScanInitializeWorker(IndexScanState *node,
node->iss_OrderByKeys, node->iss_NumOrderByKeys);
}
+/*
+ * Compute the amount of DSM we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecIndexScanInstrumentEstimate(IndexScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ /*
+ * This size calculation is trivial enough that we don't bother saving it
+ * in the IndexScanState. We'll recalculate the needed size in
+ * ExecIndexScanInstrumentInitDSM().
+ */
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel index scan instrumentation.
+ */
+void
+ExecIndexScanInstrumentInitDSM(IndexScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedIndexScanInstrumentation, winstrument) +
+ pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ node->iss_SharedInfo =
+ (SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->iss_SharedInfo, 0, size);
+ node->iss_SharedInfo->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->iss_SharedInfo);
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecIndexScanInstrumentInitWorker(IndexScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ node->iss_SharedInfo = (SharedIndexScanInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
/* ----------------------------------------------------------------
* ExecIndexScanRetrieveInstrumentation
*
diff --git a/src/include/access/genam.h b/src/include/access/genam.h
index b69320a7fc8..68bfe405db3 100644
--- a/src/include/access/genam.h
+++ b/src/include/access/genam.h
@@ -171,14 +171,9 @@ extern void index_endscan(IndexScanDesc scan);
extern void index_markpos(IndexScanDesc scan);
extern void index_restrpos(IndexScanDesc scan);
extern Size index_parallelscan_estimate(Relation indexRelation,
- int nkeys, int norderbys, Snapshot snapshot,
- bool instrument, bool parallel_aware,
- int nworkers);
+ int nkeys, int norderbys, Snapshot snapshot);
extern void index_parallelscan_initialize(Relation heapRelation,
Relation indexRelation, Snapshot snapshot,
- bool instrument, bool parallel_aware,
- int nworkers,
- SharedIndexScanInstrumentation **sharedinfo,
ParallelIndexScanDesc target);
extern void index_parallelrescan(IndexScanDesc scan);
extern IndexScanDesc index_beginscan_parallel(Relation heaprel,
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 960abf6c214..fd2076c582a 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -203,7 +203,6 @@ typedef struct ParallelIndexScanDescData
{
RelFileLocator ps_locator; /* physical table relation to scan */
RelFileLocator ps_indexlocator; /* physical index relation to scan */
- Size ps_offset_ins; /* Offset to SharedIndexScanInstrumentation */
Size ps_offset_am; /* Offset to am-specific structure */
char ps_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
} ParallelIndexScanDescData;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 2a0ff377a73..46db2dd70c4 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -18,6 +18,17 @@
#ifndef INSTRUMENT_NODE_H
#define INSTRUMENT_NODE_H
+/*
+ * Offset added to plan_node_id to create a second TOC key for per-worker scan
+ * instrumentation. This allows scan nodes to store instrumentation in a
+ * separate DSM allocation from their parallel scan descriptor. This is useful
+ * because scans may be non-parallel-aware but still require instrumentation or
+ * parallel-aware and not require instrumentation. Having two allocations
+ * provides flexibility to allocate and initialize what is needed. In the
+ * future, if nodes need more than two DSM allocations, we would need a more
+ * robust system.
+ */
+#define PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET UINT64CONST(0xD000000000000000)
/* ---------------------
* Instrumentation information for aggregate function execution
diff --git a/src/include/executor/nodeIndexonlyscan.h b/src/include/executor/nodeIndexonlyscan.h
index ba807b8d8d9..b686244eb91 100644
--- a/src/include/executor/nodeIndexonlyscan.h
+++ b/src/include/executor/nodeIndexonlyscan.h
@@ -32,6 +32,12 @@ extern void ExecIndexOnlyScanReInitializeDSM(IndexOnlyScanState *node,
ParallelContext *pcxt);
extern void ExecIndexOnlyScanInitializeWorker(IndexOnlyScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexOnlyScanInstrumentInitWorker(IndexOnlyScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecIndexOnlyScanRetrieveInstrumentation(IndexOnlyScanState *node);
#endif /* NODEINDEXONLYSCAN_H */
diff --git a/src/include/executor/nodeIndexscan.h b/src/include/executor/nodeIndexscan.h
index 803be5b08df..c60cbc9a94b 100644
--- a/src/include/executor/nodeIndexscan.h
+++ b/src/include/executor/nodeIndexscan.h
@@ -28,6 +28,12 @@ extern void ExecIndexScanInitializeDSM(IndexScanState *node, ParallelContext *pc
extern void ExecIndexScanReInitializeDSM(IndexScanState *node, ParallelContext *pcxt);
extern void ExecIndexScanInitializeWorker(IndexScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecIndexScanInstrumentEstimate(IndexScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexScanInstrumentInitDSM(IndexScanState *node,
+ ParallelContext *pcxt);
+extern void ExecIndexScanInstrumentInitWorker(IndexScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecIndexScanRetrieveInstrumentation(IndexScanState *node);
/*
--
2.43.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-06 20:39 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-06 20:39 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/6/26 18:50, Melanie Plageman wrote:
> On Mon, Apr 6, 2026 at 10:01 AM Melanie Plageman
> <melanieplageman@gmail.com> wrote:
>>
>> On Mon, Apr 6, 2026 at 5:15 AM Tomas Vondra <tomas@vondra.me> wrote:
>>>
>>> Anyway, that doesn't matter much, because the more I look at the
>>> approach with having a separate chunk of shared memory, the more I like
>>> it. It seems much simpler, more elegant, etc. I really disliked how
>>> unreadable the code got with the parallel_aware/instrument checks in
>>> multiple places, and this just gets rid of that. I like that.
>>>
>> <--snip-->
>>>
>>> Is execParallel.h the right place to define the offset? It means the
>>> various nodes (like nodeBitmapHeapScan) now have to include this header,
>>> and it seems a bit suspicious. I can't think of a better .h file, and
>>> maybe I'm wrong and it's perfectly fine.
>>
>> It could go in src/include/executor is instrument_node.h. It's where
>> the structs for the shared instrumentation go. It's probably cheaper
>> to include also because it doesn't include anything.
>
> I cleaned up the first patch in the set that refactors index-only and
> index scan to use this pattern and realized that I wasn't sure what to
> do about the duplication between index-only and index scans for these
> functions.
>
Thanks!
> ExecIndexScanInstrumentEstimate() and
> ExecIndexOnlyScanInstrumentEstimate() are the same code except they
> take different node types to do the check for intsrumentation
> if (!node->ss.ps.instrument || pcxt->nworkers == 0)
> return;
> The size estimation for SharedIndexScanIntrumentation could be common
> to the two but it is only three lines and doesn't seem worth another
> function call. ExecIndex[Only]ScanEstimate() use a common helper
> index_parallelscan_estimate() but that is more code.
Seems like a reasonable trade off to me. It's way more readable without
the conditions, even if there's a bit more code than before.
To me this seems fine.
> We could just have a common helper and only call it if
> node->ss.ps.instrument && pcxt->nworkers > 0 but I don't see anybody
> else in ExecParallelEstimate() checking for instrumentation being
> enabled.
>
Not sure I understand what helper you mean? In execParallel.c?
AFAICS now we have 3 scans that check this (index, index-only, BHS).
With the other patches there will be SeqScan and TidRangeScan too.
> Besides that, the function names are very long already but don't
> include the word parallel (though ExecIndexScanEstimate() is for
> parallel scans only and doesn't include "parallel").
>
> ExecIndex[Only]ScanInstrumentInitDSM() is the same between the two
> except the node type it takes but three different members of node are
> used here, so I'm not sure how much sense a common helper makes here.
>
> I have a feeling the above stuff isn't okay how it is now, but I'm not
> sure which direction to go.
>
> I think ExecIndex[Only]ScanInstrumentInitWorker() is fine the way it
> is because it is only 1 line and requires a different node for
> index/index-only.
>
I think it's OK. Maybe there is a scheme that makes this work both for
plain index scans and index-only scans, but I doubt it's much simpler in
the end. Doesn't seem worth it to me.
I think this is ready to go, with a tiny amount of polishing.
1) "amount of DSM" sounds a bit strange to me. The wording "amount of
space in ..." from the other nodes seems better to me. Or maybe I'm just
used to it, not sure.
2) I wonder if maybe PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET should be
placed in plannodes.h, because that's where Plan->plan_node_id is
defined. instrument_node.h works too, but the places accessing the
plan_node_id already have to include plannodes.h.
Other than this, I think it's ready to go. I went over it multiple
times, but I don't have any new comments.
We need to decide whether to push this into PG19. This was primarily
motivated by the index prefetching work, but we now know that won't
happen in PG19 :-( But the instrumentation is useful even for the three
scans using read streams, so I think it's a meaningful improvement.
If you think you can get this pushed, I'll do my best to finalize the
instrumentation and SeqScan/TidRangeScan parts.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 00:19 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-07 00:19 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 6, 2026 at 4:39 PM Tomas Vondra <tomas@vondra.me> wrote:
>
> On 4/6/26 18:50, Melanie Plageman wrote:
>
> > I cleaned up the first patch in the set that refactors index-only and
> > index scan to use this pattern and realized that I wasn't sure what to
> > do about the duplication between index-only and index scans for these
> > functions.
>
<--snip-->
>
> I think this is ready to go, with a tiny amount of polishing.
>
> 1) "amount of DSM" sounds a bit strange to me. The wording "amount of
> space in ..." from the other nodes seems better to me. Or maybe I'm just
> used to it, not sure.
Fixed that before pushing.
> 2) I wonder if maybe PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET should be
> placed in plannodes.h, because that's where Plan->plan_node_id is
> defined. instrument_node.h works too, but the places accessing the
> plan_node_id already have to include plannodes.h.
I don't think it really belongs there. It is very specific to
execution. We use the plan node id as the key for convenience, but it
isn't the purpose of plan_node_id.
PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET's only purpose is to be used
during execution for instrumentation, so it feels like it belongs in
node_instrument.h. And we don't need a separate include for it either.
It goes with the other stuff being defined in instrument_node.h (like
SharedIndexScanInstrumentation) and being used by those callers. I
admit the comment above it is a bit odd, but I think it is ultimately
okay.
> We need to decide whether to push this into PG19. This was primarily
> motivated by the index prefetching work, but we now know that won't
> happen in PG19 :-( But the instrumentation is useful even for the three
> scans using read streams, so I think it's a meaningful improvement.
I think it is a meaningful improvement too. I think we should do it.
> If you think you can get this pushed, I'll do my best to finalize the
> instrumentation and SeqScan/TidRangeScan parts.
I've pushed the first patch for index/index-only scans. Attached is
the BHS fix that uses the new pattern. It needs at least one review
before pushing. While I was polishing it, I realized I neglected to
use add_size()/mul_size() in the index-only/index scan patches. So,
0002 is just a fix commit to do that. Feel free to push these if you
think they're ready. Otherwise, I'll do so pending your review in my
morning.
- Melanie
Attachments:
[text/x-patch] v11-0001-Fix-BitmapHeapScan-non-parallel-aware-EXPLAIN-AN.patch (8.8K, ../../CAAKRu_by3fu=VsRmHjRMi9sF6Uy8mSruApyvzRm+O_=E1-ViAA@mail.gmail.com/2-v11-0001-Fix-BitmapHeapScan-non-parallel-aware-EXPLAIN-AN.patch)
download | inline diff:
From 4a792a3d864761741c7ce5fd0e10a9f1c9539094 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 16:27:22 -0400
Subject: [PATCH v11 1/2] Fix BitmapHeapScan non-parallel-aware EXPLAIN ANALYZE
Allocate shared bitmap table scan instrumentation in its own DSM chunk,
separate from ParallelBitmapHeapState. Previously, shared
instrumentation was only allocated for parallel-aware nodes, so a
non-parallel-aware bitmap table scan in a parallel query had no shared
instrumentation and EXPLAIN ANALYZE didn't report exact/lossy pages.
This affected queries like bitmap table scans on the outside of a
parallel join or those run with debug_parallel_query=regress. Fix by
allocating a separate DSM chunk for shared instrumentation and doing so
regardless of parallel-awareness.
---
src/backend/commands/explain.c | 2 +-
src/backend/executor/execParallel.c | 9 ++
src/backend/executor/nodeBitmapHeapscan.c | 111 +++++++++++++---------
src/include/executor/nodeBitmapHeapscan.h | 6 ++
4 files changed, 83 insertions(+), 45 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 73eaaf176ac..f151f21f9b3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3948,7 +3948,7 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
/* Display stats for each parallel worker */
- if (planstate->pstate != NULL)
+ if (planstate->sinstrument != NULL)
{
for (int n = 0; n < planstate->sinstrument->num_workers; n++)
{
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 726aca230a4..1a5ec0c305f 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -303,6 +303,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecBitmapHeapEstimate((BitmapHeapScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentEstimate((BitmapHeapScanState *) planstate,
+ e->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -542,6 +545,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecBitmapHeapInitializeDSM((BitmapHeapScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentInitDSM((BitmapHeapScanState *) planstate,
+ d->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -1427,6 +1433,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecBitmapHeapInitializeWorker((BitmapHeapScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentInitWorker((BitmapHeapScanState *) planstate,
+ pwcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 73831aed451..d65e2a87b42 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -495,18 +495,8 @@ void
ExecBitmapHeapEstimate(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
- Size size;
-
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
-
- /* account for instrumentation, if required */
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- {
- size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
- size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
- }
-
- shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ MAXALIGN(sizeof(ParallelBitmapHeapState)));
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -521,27 +511,15 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
ParallelBitmapHeapState *pstate;
- SharedBitmapHeapInstrumentation *sinstrument = NULL;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
- char *ptr;
- Size size;
/* If there's no DSA, there are no workers; initialize nothing. */
if (dsa == NULL)
return;
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- {
- size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
- size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
- }
-
- ptr = shm_toc_allocate(pcxt->toc, size);
- pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ pstate = (ParallelBitmapHeapState *)
+ shm_toc_allocate(pcxt->toc,
+ MAXALIGN(sizeof(ParallelBitmapHeapState)));
pstate->tbmiterator = 0;
@@ -551,18 +529,8 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ConditionVariableInit(&pstate->cv);
- if (sinstrument)
- {
- sinstrument->num_workers = pcxt->nworkers;
-
- /* ensure any unfilled slots will contain zeroes */
- memset(sinstrument->sinstrument, 0,
- pcxt->nworkers * sizeof(BitmapHeapScanInstrumentation));
- }
-
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pstate);
node->pstate = pstate;
- node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -600,17 +568,72 @@ void
ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
- char *ptr;
-
Assert(node->ss.ps.state->es_query_dsa != NULL);
- ptr = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ node->pstate = (ParallelBitmapHeapState *)
+ shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+}
+
+/*
+ * Compute the amount of space we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecBitmapHeapInstrumentEstimate(BitmapHeapScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedBitmapHeapInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel bitmap heap scan instrumentation.
+ */
+void
+ExecBitmapHeapInstrumentInitDSM(BitmapHeapScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedBitmapHeapInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
+ node->sinstrument =
+ (SharedBitmapHeapInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->sinstrument, 0, size);
+ node->sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->sinstrument);
+}
- node->pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecBitmapHeapInstrumentInitWorker(BitmapHeapScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.instrument)
+ return;
- if (node->ss.ps.instrument)
- node->sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ node->sinstrument = (SharedBitmapHeapInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index 5d82f71abff..5c0b6592a1f 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,6 +28,12 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecBitmapHeapInstrumentEstimate(BitmapHeapScanState *node,
+ ParallelContext *pcxt);
+extern void ExecBitmapHeapInstrumentInitDSM(BitmapHeapScanState *node,
+ ParallelContext *pcxt);
+extern void ExecBitmapHeapInstrumentInitWorker(BitmapHeapScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecBitmapHeapRetrieveInstrumentation(BitmapHeapScanState *node);
#endif /* NODEBITMAPHEAPSCAN_H */
--
2.43.0
[text/x-patch] v11-0002-Use-add_size-mul_size-for-index-instrumentation-.patch (3.1K, ../../CAAKRu_by3fu=VsRmHjRMi9sF6Uy8mSruApyvzRm+O_=E1-ViAA@mail.gmail.com/3-v11-0002-Use-add_size-mul_size-for-index-instrumentation-.patch)
download | inline diff:
From 9ab70837dac963dc1ae6721d0f45dc2c9d3f398d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 6 Apr 2026 20:06:49 -0400
Subject: [PATCH v11 2/2] Use add_size/mul_size for index instrumentation size
calculations
Use overflow-safe size arithmetic in the Index[Only]Scan and parallel
instrumentation functions, consistent with other executor nodes (Hash,
Sort, Agg, Memoize). This was an oversight in dd78e69cfc3.
---
src/backend/executor/nodeIndexonlyscan.c | 8 ++++----
src/backend/executor/nodeIndexscan.c | 8 ++++----
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c4ff422756e..d52012e8a69 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -854,8 +854,8 @@ ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
* in the IndexOnlyScanState. We'll recalculate the needed size in
* ExecIndexOnlyScanInstrumentInitDSM().
*/
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -872,8 +872,8 @@ ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
if (!node->ss.ps.instrument || pcxt->nworkers == 0)
return;
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
node->ioss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 6cc927f2454..39f6691ee35 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -1789,8 +1789,8 @@ ExecIndexScanInstrumentEstimate(IndexScanState *node,
* in the IndexScanState. We'll recalculate the needed size in
* ExecIndexScanInstrumentInitDSM().
*/
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -1807,8 +1807,8 @@ ExecIndexScanInstrumentInitDSM(IndexScanState *node,
if (!node->ss.ps.instrument || pcxt->nworkers == 0)
return;
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
node->iss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
--
2.43.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-07 01:14 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-07 01:14 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/7/26 02:19, Melanie Plageman wrote:
> On Mon, Apr 6, 2026 at 4:39 PM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> On 4/6/26 18:50, Melanie Plageman wrote:
>>
>>> I cleaned up the first patch in the set that refactors index-only and
>>> index scan to use this pattern and realized that I wasn't sure what to
>>> do about the duplication between index-only and index scans for these
>>> functions.
>>
> <--snip-->
>>
>> I think this is ready to go, with a tiny amount of polishing.
>>
>> 1) "amount of DSM" sounds a bit strange to me. The wording "amount of
>> space in ..." from the other nodes seems better to me. Or maybe I'm just
>> used to it, not sure.
>
> Fixed that before pushing.
>
>> 2) I wonder if maybe PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET should be
>> placed in plannodes.h, because that's where Plan->plan_node_id is
>> defined. instrument_node.h works too, but the places accessing the
>> plan_node_id already have to include plannodes.h.
>
> I don't think it really belongs there. It is very specific to
> execution. We use the plan node id as the key for convenience, but it
> isn't the purpose of plan_node_id.
> PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET's only purpose is to be used
> during execution for instrumentation, so it feels like it belongs in
> node_instrument.h. And we don't need a separate include for it either.
> It goes with the other stuff being defined in instrument_node.h (like
> SharedIndexScanInstrumentation) and being used by those callers. I
> admit the comment above it is a bit odd, but I think it is ultimately
> okay.
>
WFM
>> We need to decide whether to push this into PG19. This was primarily
>> motivated by the index prefetching work, but we now know that won't
>> happen in PG19 :-( But the instrumentation is useful even for the three
>> scans using read streams, so I think it's a meaningful improvement.
>
> I think it is a meaningful improvement too. I think we should do it.
OK
>> If you think you can get this pushed, I'll do my best to finalize the
>> instrumentation and SeqScan/TidRangeScan parts.
>
> I've pushed the first patch for index/index-only scans. Attached is
> the BHS fix that uses the new pattern. It needs at least one review
> before pushing. While I was polishing it, I realized I neglected to
> use add_size()/mul_size() in the index-only/index scan patches. So,
> 0002 is just a fix commit to do that. Feel free to push these if you
> think they're ready. Otherwise, I'll do so pending your review in my
> morning.
>
Thanks!
I'll take a look in my morning, and will consider pushing the changes so
that I can start pushing the parts adding the new instrumentation.
I've spent a fair amount of time looking at those parts today. Attached
is v12 (including the two parts you posted as v11), with two or three
bigger changes compared to earlier versions:
1) default OFF
After thinking about it a bit more, I decided to change the default to
OFF. On the one hand I agree it's somewhat similar to BUFFERS, and that
option is ON by default now. But on the other hand, we must not clutter
EXPLAIN output with too much information, and I'm not convinced IO is
worth it. So I changed to OFF by default. That also makes the patches
smaller, due to not having to adjust that many tests.
2) auto_explain
It also occurred to me we should add a matching option to auto_explain,
similarly to log_buffers. 0005 does that.
3) INSTRUMENT_IO
The auto_explain bit also implies we need a new INSTRUMENT_IO constant,
to handle this just like BUFFERS. Which also means we can actually
collect the stats only when the IO option is specified (instead of for
all ANALYZE runs). Which is nice.
I plan to do some more testing in my morning, add missing comments,
polish the commit messages etc. And then eventually push it.
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v12-0001-Fix-BitmapHeapScan-non-parallel-aware-EXPLAIN-AN.patch (8.8K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/2-v12-0001-Fix-BitmapHeapScan-non-parallel-aware-EXPLAIN-AN.patch)
download | inline diff:
From e7b4396be97e49a9c1791ce12de268229575bd83 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 16:27:22 -0400
Subject: [PATCH v12 1/7] Fix BitmapHeapScan non-parallel-aware EXPLAIN ANALYZE
Allocate shared bitmap table scan instrumentation in its own DSM chunk,
separate from ParallelBitmapHeapState. Previously, shared
instrumentation was only allocated for parallel-aware nodes, so a
non-parallel-aware bitmap table scan in a parallel query had no shared
instrumentation and EXPLAIN ANALYZE didn't report exact/lossy pages.
This affected queries like bitmap table scans on the outside of a
parallel join or those run with debug_parallel_query=regress. Fix by
allocating a separate DSM chunk for shared instrumentation and doing so
regardless of parallel-awareness.
---
src/backend/commands/explain.c | 2 +-
src/backend/executor/execParallel.c | 9 ++
src/backend/executor/nodeBitmapHeapscan.c | 111 +++++++++++++---------
src/include/executor/nodeBitmapHeapscan.h | 6 ++
4 files changed, 83 insertions(+), 45 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 73eaaf176ac..f151f21f9b3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3948,7 +3948,7 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
/* Display stats for each parallel worker */
- if (planstate->pstate != NULL)
+ if (planstate->sinstrument != NULL)
{
for (int n = 0; n < planstate->sinstrument->num_workers; n++)
{
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 726aca230a4..1a5ec0c305f 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -303,6 +303,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecBitmapHeapEstimate((BitmapHeapScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentEstimate((BitmapHeapScanState *) planstate,
+ e->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -542,6 +545,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecBitmapHeapInitializeDSM((BitmapHeapScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentInitDSM((BitmapHeapScanState *) planstate,
+ d->pcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
@@ -1427,6 +1433,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecBitmapHeapInitializeWorker((BitmapHeapScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecBitmapHeapInstrumentInitWorker((BitmapHeapScanState *) planstate,
+ pwcxt);
break;
case T_HashJoinState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 73831aed451..d65e2a87b42 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -495,18 +495,8 @@ void
ExecBitmapHeapEstimate(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
- Size size;
-
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
-
- /* account for instrumentation, if required */
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- {
- size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
- size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
- }
-
- shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ MAXALIGN(sizeof(ParallelBitmapHeapState)));
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -521,27 +511,15 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
ParallelBitmapHeapState *pstate;
- SharedBitmapHeapInstrumentation *sinstrument = NULL;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
- char *ptr;
- Size size;
/* If there's no DSA, there are no workers; initialize nothing. */
if (dsa == NULL)
return;
- size = MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- {
- size = add_size(size, offsetof(SharedBitmapHeapInstrumentation, sinstrument));
- size = add_size(size, mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
- }
-
- ptr = shm_toc_allocate(pcxt->toc, size);
- pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
- if (node->ss.ps.instrument && pcxt->nworkers > 0)
- sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ pstate = (ParallelBitmapHeapState *)
+ shm_toc_allocate(pcxt->toc,
+ MAXALIGN(sizeof(ParallelBitmapHeapState)));
pstate->tbmiterator = 0;
@@ -551,18 +529,8 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ConditionVariableInit(&pstate->cv);
- if (sinstrument)
- {
- sinstrument->num_workers = pcxt->nworkers;
-
- /* ensure any unfilled slots will contain zeroes */
- memset(sinstrument->sinstrument, 0,
- pcxt->nworkers * sizeof(BitmapHeapScanInstrumentation));
- }
-
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pstate);
node->pstate = pstate;
- node->sinstrument = sinstrument;
}
/* ----------------------------------------------------------------
@@ -600,17 +568,72 @@ void
ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
- char *ptr;
-
Assert(node->ss.ps.state->es_query_dsa != NULL);
- ptr = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+ node->pstate = (ParallelBitmapHeapState *)
+ shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
+}
+
+/*
+ * Compute the amount of space we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecBitmapHeapInstrumentEstimate(BitmapHeapScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedBitmapHeapInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel bitmap heap scan instrumentation.
+ */
+void
+ExecBitmapHeapInstrumentInitDSM(BitmapHeapScanState *node,
+ ParallelContext *pcxt)
+{
+ Size size;
+
+ if (!node->ss.ps.instrument || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedBitmapHeapInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(BitmapHeapScanInstrumentation)));
+ node->sinstrument =
+ (SharedBitmapHeapInstrumentation *) shm_toc_allocate(pcxt->toc, size);
+
+ /* Each per-worker area must start out as zeroes */
+ memset(node->sinstrument, 0, size);
+ node->sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ node->sinstrument);
+}
- node->pstate = (ParallelBitmapHeapState *) ptr;
- ptr += MAXALIGN(sizeof(ParallelBitmapHeapState));
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecBitmapHeapInstrumentInitWorker(BitmapHeapScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.instrument)
+ return;
- if (node->ss.ps.instrument)
- node->sinstrument = (SharedBitmapHeapInstrumentation *) ptr;
+ node->sinstrument = (SharedBitmapHeapInstrumentation *)
+ shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
}
/* ----------------------------------------------------------------
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index 5d82f71abff..5c0b6592a1f 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,6 +28,12 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+extern void ExecBitmapHeapInstrumentEstimate(BitmapHeapScanState *node,
+ ParallelContext *pcxt);
+extern void ExecBitmapHeapInstrumentInitDSM(BitmapHeapScanState *node,
+ ParallelContext *pcxt);
+extern void ExecBitmapHeapInstrumentInitWorker(BitmapHeapScanState *node,
+ ParallelWorkerContext *pwcxt);
extern void ExecBitmapHeapRetrieveInstrumentation(BitmapHeapScanState *node);
#endif /* NODEBITMAPHEAPSCAN_H */
--
2.53.0
[text/x-patch] v12-0002-Use-add_size-mul_size-for-index-instrumentation-.patch (3.1K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/3-v12-0002-Use-add_size-mul_size-for-index-instrumentation-.patch)
download | inline diff:
From eb1b6b6f483d6ca77e869fd81cecd246cca3bf53 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 6 Apr 2026 20:06:49 -0400
Subject: [PATCH v12 2/7] Use add_size/mul_size for index instrumentation size
calculations
Use overflow-safe size arithmetic in the Index[Only]Scan and parallel
instrumentation functions, consistent with other executor nodes (Hash,
Sort, Agg, Memoize). This was an oversight in dd78e69cfc3.
---
src/backend/executor/nodeIndexonlyscan.c | 8 ++++----
src/backend/executor/nodeIndexscan.c | 8 ++++----
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c
index c4ff422756e..d52012e8a69 100644
--- a/src/backend/executor/nodeIndexonlyscan.c
+++ b/src/backend/executor/nodeIndexonlyscan.c
@@ -854,8 +854,8 @@ ExecIndexOnlyScanInstrumentEstimate(IndexOnlyScanState *node,
* in the IndexOnlyScanState. We'll recalculate the needed size in
* ExecIndexOnlyScanInstrumentInitDSM().
*/
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -872,8 +872,8 @@ ExecIndexOnlyScanInstrumentInitDSM(IndexOnlyScanState *node,
if (!node->ss.ps.instrument || pcxt->nworkers == 0)
return;
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
node->ioss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
diff --git a/src/backend/executor/nodeIndexscan.c b/src/backend/executor/nodeIndexscan.c
index 6cc927f2454..39f6691ee35 100644
--- a/src/backend/executor/nodeIndexscan.c
+++ b/src/backend/executor/nodeIndexscan.c
@@ -1789,8 +1789,8 @@ ExecIndexScanInstrumentEstimate(IndexScanState *node,
* in the IndexScanState. We'll recalculate the needed size in
* ExecIndexScanInstrumentInitDSM().
*/
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
shm_toc_estimate_chunk(&pcxt->estimator, size);
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -1807,8 +1807,8 @@ ExecIndexScanInstrumentInitDSM(IndexScanState *node,
if (!node->ss.ps.instrument || pcxt->nworkers == 0)
return;
- size = offsetof(SharedIndexScanInstrumentation, winstrument) +
- pcxt->nworkers * sizeof(IndexScanInstrumentation);
+ size = add_size(offsetof(SharedIndexScanInstrumentation, winstrument),
+ mul_size(pcxt->nworkers, sizeof(IndexScanInstrumentation)));
node->iss_SharedInfo =
(SharedIndexScanInstrumentation *) shm_toc_allocate(pcxt->toc, size);
--
2.53.0
[text/x-patch] v12-0003-switch-explain-to-unaligned-for-json-xml-yaml.patch (12.9K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/4-v12-0003-switch-explain-to-unaligned-for-json-xml-yaml.patch)
download | inline diff:
From b8561cc9b76282a0199a501c5b82daf093c91987 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v12 3/7] switch explain to unaligned for json/xml/yaml
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.53.0
[text/x-patch] v12-0004-Add-EXPLAIN-IO-infrastructure-and-BitmapHeapScan.patch (19.8K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/5-v12-0004-Add-EXPLAIN-IO-infrastructure-and-BitmapHeapScan.patch)
download | inline diff:
From b584342347342bc0db8a96ea99094dbe8f48c7d3 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:17:22 -0400
Subject: [PATCH v12 4/7] Add EXPLAIN (IO) infrastructure and BitmapHeapScan IO
instrumentation
---
doc/src/sgml/ref/explain.sgml | 12 +++
src/backend/access/heap/heapam.c | 10 ++
src/backend/commands/explain.c | 112 ++++++++++++++++++++++
src/backend/commands/explain_state.c | 8 ++
src/backend/executor/nodeBitmapHeapscan.c | 19 +++-
src/backend/storage/aio/read_stream.c | 87 +++++++++++++++++
src/include/access/relscan.h | 6 ++
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument.h | 1 +
src/include/executor/instrument_node.h | 50 ++++++++++
src/include/storage/read_stream.h | 2 +
src/tools/pgindent/typedefs.list | 2 +
13 files changed, 311 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..a854c41e963 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>FALSE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index f6ac5a0897c..89ab9742aa5 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -43,6 +43,7 @@
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/instrument_node.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/lmgr.h"
@@ -1200,6 +1201,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1312,6 +1314,14 @@ heap_beginscan(Relation relation, Snapshot snapshot,
sizeof(TBMIterateResult));
}
+ /* enable read stream instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ {
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ read_stream_enable_stats(scan->rs_read_stream,
+ &scan->rs_base.rs_instrument->io);
+ }
+
scan->rs_vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f151f21f9b3..863a9dd0f0d 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,7 @@
*/
#include "postgres.h"
+#include "access/relscan.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +140,8 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -519,6 +522,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
instrument_option |= INSTRUMENT_BUFFERS;
if (es->wal)
instrument_option |= INSTRUMENT_WAL;
+ else if (es->io)
+ instrument_option |= INSTRUMENT_IO;
/*
* We always collect timing for the entire statement, even when node-level
@@ -2008,6 +2013,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -3984,6 +3990,112 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f inprogress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats = {0};
+
+ if (!es->io)
+ return;
+
+ /*
+ * Initialize counters with stats from the local process first.
+ *
+ * The scan descriptor may not exist, e.g. if the scan did not start, or
+ * because of debug_parallel_query=regress. We still want to collect data
+ * from workers.
+ */
+ if (planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument)
+ {
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+ }
+
+ /*
+ * Accumulate data from parallel workers (if any).
+ */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 65dd4111459..0e07a63fca6 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -162,6 +162,8 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ es->io = defGetBoolean(opt);
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -188,6 +190,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d65e2a87b42..83d6478bc2b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -144,13 +144,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -330,6 +337,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect I/O instrumentation for this process */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 0b6cdf7c873..936e08ea450 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -74,6 +74,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -123,6 +124,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -188,6 +192,73 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+/*
+ * Update stream stats about waits for I/O when consuming buffers.
+ *
+ * We count the number of I/O waits while pulling buffers out of a stream.
+ */
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
+/*
+ * Enable collection of stats into the provided IOStats.
+ */
+void
+read_stream_enable_stats(ReadStream *stream, IOStats *stats)
+{
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = stream->max_pinned_buffers;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -426,6 +497,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -1021,6 +1095,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -1036,6 +1111,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
* to avoid having to effectively do another synchronous IO for
* the next block (if it were also a miss).
*/
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -1100,6 +1181,10 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+ /* Count it as a wait if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
+
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
if (++stream->oldest_io_index == stream->max_ios)
@@ -1228,6 +1313,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fd2076c582a..2ea06a67a63 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,7 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -64,6 +65,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a21c7db5439..c13f05d39db 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -69,6 +69,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 6252fe11f15..97bc7ed49f6 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument.h b/src/include/executor/instrument.h
index cc9fbb0e2f0..f093a52aae0 100644
--- a/src/include/executor/instrument.h
+++ b/src/include/executor/instrument.h
@@ -64,6 +64,7 @@ typedef enum InstrumentOption
INSTRUMENT_BUFFERS = 1 << 1, /* needs buffer usage */
INSTRUMENT_ROWS = 1 << 2, /* needs row count */
INSTRUMENT_WAL = 1 << 3, /* needs WAL usage */
+ INSTRUMENT_IO = 1 << 4, /* needs IO usage */
INSTRUMENT_ALL = PG_INT32_MAX
} InstrumentOption;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index e6a3f9f1941..22a75ccd863 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -48,6 +48,55 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams and I/O
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
@@ -79,6 +128,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..aebb1fafb31 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+typedef struct IOStats IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -103,5 +104,6 @@ extern BlockNumber read_stream_pause(ReadStream *stream);
extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern void read_stream_enable_stats(ReadStream *stream, IOStats *stats);
#endif /* READ_STREAM_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9e6a39f5608..98b8d78e693 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1279,6 +1279,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3127,6 +3128,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.53.0
[text/x-patch] v12-0005-auto_explain.patch (3.3K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/6-v12-0005-auto_explain.patch)
download | inline diff:
From 85990c9c99327e76a5ede213bb705d33d97a1047 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Tue, 7 Apr 2026 02:04:06 +0200
Subject: [PATCH v12 5/7] auto_explain
---
contrib/auto_explain/auto_explain.c | 15 +++++++++++++++
doc/src/sgml/auto-explain.sgml | 20 ++++++++++++++++++++
2 files changed, 35 insertions(+)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 6ceae1c69ce..2eb2369c354 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -38,6 +38,7 @@ static int auto_explain_log_parameter_max_length = -1; /* bytes or -1 */
static bool auto_explain_log_analyze = false;
static bool auto_explain_log_verbose = false;
static bool auto_explain_log_buffers = false;
+static bool auto_explain_log_io = false;
static bool auto_explain_log_wal = false;
static bool auto_explain_log_triggers = false;
static bool auto_explain_log_timing = true;
@@ -203,6 +204,17 @@ _PG_init(void)
NULL,
NULL);
+ DefineCustomBoolVariable("auto_explain.log_io",
+ "Log IO statistics.",
+ NULL,
+ &auto_explain_log_io,
+ false,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
DefineCustomBoolVariable("auto_explain.log_wal",
"Log WAL usage.",
NULL,
@@ -343,6 +355,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
queryDesc->instrument_options |= INSTRUMENT_ROWS;
if (auto_explain_log_buffers)
queryDesc->instrument_options |= INSTRUMENT_BUFFERS;
+ if (auto_explain_log_io)
+ queryDesc->instrument_options |= INSTRUMENT_IO;
if (auto_explain_log_wal)
queryDesc->instrument_options |= INSTRUMENT_WAL;
}
@@ -440,6 +454,7 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->analyze = (queryDesc->instrument_options && auto_explain_log_analyze);
es->verbose = auto_explain_log_verbose;
es->buffers = (es->analyze && auto_explain_log_buffers);
+ es->io = (es->analyze && auto_explain_log_io);
es->wal = (es->analyze && auto_explain_log_wal);
es->timing = (es->analyze && auto_explain_log_timing);
es->summary = es->analyze;
diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index ee85a67eb2e..06a8fcc6c5b 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -128,6 +128,26 @@ LOAD 'auto_explain';
</listitem>
</varlistentry>
+ <varlistentry id="auto-explain-configuration-parameters-log-io">
+ <term>
+ <varname>auto_explain.log_io</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>auto_explain.log_io</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <varname>auto_explain.log_io</varname> controls whether I/O usage
+ statistics are printed when an execution plan is logged; it's
+ equivalent to the <literal>IO</literal> option of <command>EXPLAIN</command>.
+ This parameter has no effect
+ unless <varname>auto_explain.log_analyze</varname> is enabled.
+ This parameter is off by default.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="auto-explain-configuration-parameters-log-wal">
<term>
<varname>auto_explain.log_wal</varname> (<type>boolean</type>)
--
2.53.0
[text/x-patch] v12-0006-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch (14.0K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/7-v12-0006-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch)
download | inline diff:
From 03899df55da0e525668ad6ef0d37fcc18ecf5dab Mon Sep 17 00:00:00 2001
From: test <test>
Date: Tue, 7 Apr 2026 01:38:53 +0200
Subject: [PATCH v12 6/7] Add EXPLAIN (IO) instrumentation for SeqScan
---
src/backend/commands/explain.c | 25 ++++++
src/backend/executor/execParallel.c | 11 +++
src/backend/executor/nodeSeqscan.c | 116 +++++++++++++++++++++++--
src/include/executor/instrument_node.h | 19 ++++
src/include/executor/nodeSeqscan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/test/regress/expected/explain.out | 18 +++-
src/test/regress/sql/explain.sql | 4 +-
src/tools/pgindent/typedefs.list | 2 +
9 files changed, 192 insertions(+), 13 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 863a9dd0f0d..ac1bbf19a80 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2032,6 +2032,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -4086,6 +4087,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 1a5ec0c305f..9690f0938ae 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -257,6 +257,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecSeqScanEstimate((SeqScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentEstimate((SeqScanState *) planstate,
+ e->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -500,6 +503,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeDSM((SeqScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitDSM((SeqScanState *) planstate,
+ d->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -1148,6 +1154,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
@@ -1388,6 +1397,8 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
case T_SeqScanState:
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..b95ac2a8696 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -29,6 +29,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
+#include "executor/execParallel.h"
#include "executor/execScan.h"
#include "executor/executor.h"
#include "executor/nodeSeqscan.h"
@@ -65,15 +66,21 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
- 0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ 0, NULL, flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -297,6 +304,24 @@ ExecEndSeqScan(SeqScanState *node)
{
TableScanDesc scanDesc;
+ /*
+ * Collect IO stats for this process into shared instrumentation.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber <= node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
+ }
+
/*
* get information from node
*/
@@ -370,6 +395,13 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -378,9 +410,7 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
}
/* ----------------------------------------------------------------
@@ -410,10 +440,78 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+}
+
+void
+ExecSeqScanInstrumentEstimate(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ offsetof(SharedSeqScanInstrumentation, sinstrument) +
+ sizeof(SeqScanInstrumentation) * pcxt->nworkers);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecSeqScanInstrumentInitDSM(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedSeqScanInstrumentation *sinstrument;
+ Size size;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument) +
+ sizeof(SeqScanInstrumentation) * pcxt->nworkers;
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->sinstrument = sinstrument;
+}
+
+void
+ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.state->es_instrument)
+ return;
+
+ node->sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 22a75ccd863..003dc262b5d 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -266,4 +266,23 @@ typedef struct SharedIncrementalSortInfo
IncrementalSortInfo sinfo[FLEXIBLE_ARRAY_MEMBER];
} SharedIncrementalSortInfo;
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..9c0ad4879d7 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -28,4 +28,13 @@ extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecSeqScanInstrumentEstimate(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitDSM(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
+
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 3ecae7552fc..56febb3204c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1670,6 +1670,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ struct SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..74a4d87801e 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -100,7 +100,7 @@ select explain_filter('explain (buffers, format text) select * from int8_tbl i8'
(1 row)
\a
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
explain_filter
<explain xmlns="http://www.postgresql.org/N/explain">
<Query>
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -149,7 +156,7 @@ explain_filter
</Query>
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
explain_filter
- Plan:
Node Type: "Seq Scan"
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..2f163c64bf6 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -69,8 +69,8 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
\a
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 98b8d78e693..c0436a13ac3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2800,6 +2800,7 @@ SelfJoinCandidate
SemTPadded
SemiAntiJoinFactors
SeqScan
+SeqScanInstrumentation
SeqScanState
SeqTable
SeqTableData
@@ -2864,6 +2865,7 @@ SharedMemoizeInfo
SharedRecordTableEntry
SharedRecordTableKey
SharedRecordTypmodRegistry
+SharedSeqScanInstrumentation
SharedSortInfo
SharedTuplestore
SharedTuplestoreAccessor
--
2.53.0
[text/x-patch] v12-0007-Add-EXPLAIN-IO-instrumentation-for-TidRangeScan.patch (11.6K, ../../8d6187c9-62ac-43a3-88d1-7503c4206657@vondra.me/8-v12-0007-Add-EXPLAIN-IO-instrumentation-for-TidRangeScan.patch)
download | inline diff:
From bb6eb339595f5e21e05ff12fe7260aace3b7e564 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:31:18 -0400
Subject: [PATCH v12 7/7] Add EXPLAIN (IO) instrumentation for TidRangeScan
---
src/backend/commands/explain.c | 25 ++++++
src/backend/executor/execParallel.c | 12 +++
src/backend/executor/nodeTidrangescan.c | 113 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 18 ++++
src/include/executor/nodeTidrangescan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 172 insertions(+), 8 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ac1bbf19a80..257ce2bfc54 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2149,6 +2149,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4111,6 +4112,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9690f0938ae..81b87d82fab 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -291,6 +291,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentEstimate((TidRangeScanState *) planstate,
+ e->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -536,6 +539,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitDSM((TidRangeScanState *) planstate,
+ d->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -1157,6 +1163,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
@@ -1430,6 +1439,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitWorker((TidRangeScanState *) planstate,
+ pwcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..012e88e88f6 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -18,8 +18,10 @@
#include "access/sysattr.h"
#include "access/tableam.h"
#include "catalog/pg_operator.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
#include "executor/nodeTidrangescan.h"
+#include "executor/instrument.h"
#include "nodes/nodeFuncs.h"
#include "utils/rel.h"
@@ -242,12 +244,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +351,19 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /* Collect IO stats for this process into shared instrumentation */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber <= node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (scan && scan->rs_instrument)
+ AccumulateIOStats(&si->stats.io,
+ &scan->rs_instrument->io);
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -454,6 +476,13 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -462,9 +491,7 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -494,11 +521,81 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+}
+
+void
+ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ offsetof(SharedTidRangeScanInstrumentation, sinstrument) +
+ sizeof(TidRangeScanInstrumentation) * pcxt->nworkers);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+void
+ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedTidRangeScanInstrumentation *sinstrument;
+ Size size;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument) +
+ sizeof(TidRangeScanInstrumentation) * pcxt->nworkers;
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->trss_sinstrument = sinstrument;
+}
+
+void
+ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.state->es_instrument)
+ return;
+
+ node->trss_sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 003dc262b5d..4076990408e 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -285,4 +285,22 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+
+/*
+ * Instrumentation information for TID range scans
+ */
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..9e7d0a357bb 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -28,4 +28,13 @@ extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelConte
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
+
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 56febb3204c..13359180d25 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1922,6 +1922,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ struct SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c0436a13ac3..0cd41106975 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2867,6 +2867,7 @@ SharedRecordTableKey
SharedRecordTypmodRegistry
SharedSeqScanInstrumentation
SharedSortInfo
+SharedTidRangeScanInstrumentation
SharedTuplestore
SharedTuplestoreAccessor
SharedTuplestoreChunk
@@ -3171,6 +3172,7 @@ TidOpExpr
TidPath
TidRangePath
TidRangeScan
+TidRangeScanInstrumentation
TidRangeScanState
TidScan
TidScanState
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 08:00 ` Lukas Fittl <lukas@fittl.com>
2026-04-07 08:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 15:09 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 3 replies; 59+ messages in thread
From: Lukas Fittl @ 2026-04-07 08:00 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 6, 2026 at 6:14 PM Tomas Vondra <tomas@vondra.me> wrote:
>
> On 4/7/26 02:19, Melanie Plageman wrote:
> > On Mon, Apr 6, 2026 at 4:39 PM Tomas Vondra <tomas@vondra.me> wrote:
> >>
> >> On 4/6/26 18:50, Melanie Plageman wrote:
> >> If you think you can get this pushed, I'll do my best to finalize the
> >> instrumentation and SeqScan/TidRangeScan parts.
> >
> > I've pushed the first patch for index/index-only scans. Attached is
> > the BHS fix that uses the new pattern. It needs at least one review
> > before pushing. While I was polishing it, I realized I neglected to
> > use add_size()/mul_size() in the index-only/index scan patches. So,
> > 0002 is just a fix commit to do that. Feel free to push these if you
> > think they're ready. Otherwise, I'll do so pending your review in my
> > morning.
> >
>
> Thanks!
>
> I'll take a look in my morning, and will consider pushing the changes so
> that I can start pushing the parts adding the new instrumentation.
FWIW, 0001 and 0002 in the v12 patch set also make sense to me from a
code review just now.
>
> I've spent a fair amount of time looking at those parts today. Attached
> is v12 (including the two parts you posted as v11)
0003 looks good, but commit message needs improvement / justification
of the change (I assume you had already planned that).
0004:
> diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
> index 5b8b521802e..a854c41e963 100644
> --- a/doc/src/sgml/ref/explain.sgml
> +++ b/doc/src/sgml/ref/explain.sgml
> @@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
> TIMING [ <replaceable class="parameter">boolean</replaceable> ]
> SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
> MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
> + IO [ <replaceable class="parameter">boolean</replaceable> ]
> FORMAT { TEXT | XML | JSON | YAML }
> </synopsis>
> </refsynopsisdiv>
> @@ -298,6 +299,17 @@ ROLLBACK;
> </listitem>
> </varlistentry>
> + <varlistentry>
> + <term><literal>IO</literal></term>
> + <listitem>
> + <para>
> + Include information on I/O performed by each node.
> + This parameter may only be used when <literal>ANALYZE</literal> is also
> + enabled. It defaults to <literal>FALSE</literal>.
> + </para>
> + </listitem>
> + </varlistentry>
+
Assuming my understanding of where this works is correct, this needs
to be qualified I think in terms of which nodes get I/O activity
shown. Otherwise I as the user will expect e.g. an Index Scan to show
I/O statistics. Additionally I think this should clearly say "read
I/O", since writes I/O information wouldn't show for any nodes.
> diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
> index f151f21f9b3..863a9dd0f0d 100644
> --- a/src/backend/commands/explain.c
> +++ b/src/backend/commands/explain.c
> @@ -3984,6 +3990,112 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
> }
> }
> ...
> + /* prefetch I/O info (only if there were actual I/Os) */
> + if (stats->io_count > 0)
> + {
> + ExplainIndentText(es);
> + appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
> + " size=%.2f inprogress=%.2f\n",
> + stats->io_count, stats->wait_count,
> + (stats->io_nblocks * 1.0 / stats->io_count),
> + (stats->io_in_progress * 1.0 / stats->io_count));
> + }
I've briefly pondered again whether "I/O:" is a bit too broad here in
EXPLAIN text output, i.e. whether it should qualified it somehow, e.g.
"I/O Stream", since its just the read stream I/O activity.
Right now it looks like this:
Seq Scan on xxx (cost=0.00..7.14 rows=114 width=483) (actual
time=0.140..0.222 rows=114.00 loops=1)
Prefetch: avg=1.67 max=3 capacity=94
I/O: count=3 waits=3 size=2.00 inprogress=1.00
Buffers: shared read=6
I/O Timings: shared read=0.158
What if we did this instead:
Seq Scan on xxx (cost=0.00..7.14 rows=114 width=483) (actual
time=0.140..0.222 rows=114.00 loops=1)
Prefetch: avg=1.67 max=3 capacity=94
I/O Stream: count=3 waits=3 size=2.00 inprogress=1.00
Buffers: shared read=6
I/O Timings: shared read=0.158
But I'm 50/50 whether that's a big enough problem to debate right now.
Especially with index prefetching out of the picture this is actually
less of a problem, since this should only show on scans where the I/O
activity is mainly (fully?) relying on read streams. If we had index
prefetching, my understanding is we would have had "I/O" show the read
stream activity (i.e. table access), but not the index access,
correct?
Btw, in that same test, when I re-run I get Prefetch but not I/O -- why is that?
Seq Scan on organizations (cost=0.00..7.42 rows=1 width=483) (actual
time=0.034..0.042 rows=1.00 loops=1)
Filter: (slug = 'pganalyze'::text)
Rows Removed by Filter: 113
Prefetch: avg=1.00 max=1 capacity=94
Buffers: shared hit=6
(if its all buffers hit, why are there any prefetches at all?)
> with two or three
> bigger changes compared to earlier versions:
>
> 1) default OFF
>
> After thinking about it a bit more, I decided to change the default to
> OFF. On the one hand I agree it's somewhat similar to BUFFERS, and that
> option is ON by default now. But on the other hand, we must not clutter
> EXPLAIN output with too much information, and I'm not convinced IO is
> worth it. So I changed to OFF by default. That also makes the patches
> smaller, due to not having to adjust that many tests.
Seems fair in terms of what to do now.
>
> 2) auto_explain
>
> It also occurred to me we should add a matching option to auto_explain,
> similarly to log_buffers. 0005 does that.
Agreed.
>
> 3) INSTRUMENT_IO
>
> The auto_explain bit also implies we need a new INSTRUMENT_IO constant,
> to handle this just like BUFFERS. Which also means we can actually
> collect the stats only when the IO option is specified (instead of for
> all ANALYZE runs). Which is nice.
FWIW, it'd be the only kind that's not populated by instrument.c (and
isn't a part of Instrumentation), but I don't see any immediate
practical issues with having the extra flag.
It does mean that INSTRUMENT_ALL will turn on INSTRUMENT_IO though for
the Instrumentation, which will always be a no-op. In practice the
only users of that are the InstrAlloc calls in
pg_stat_statements/auto_explain, which I have a patch for to refactor
anyway (over on the stack-based instrumentation thread, which to be
clear is PG20 material at this point).
We could probably just drop INSTRUMENT_ALL in the next cycle to avoid
confusion. For context, that use of INSTRUMENT_ALL won't affect
es_instrument, just how queryDesc->totaltime is initialized, and what
it captures at a summary level - and it won't know how to capture I/O
stats since they are not part of Instrumentation.
I would suggest noting the fact that its a no-op for Instrumentation
itself somewhere in code at least, e.g. where INSTRUMENT_IO is defined
on InstrumentOption.
0005 looks good.
I've skimmed 0006 and 0007 and it makes sense to me why they exist
(and we should add them), but I haven't reviewed the code for
correctness.
Thanks,
Lukas
--
Lukas Fittl
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
@ 2026-04-07 08:05 ` Lukas Fittl <lukas@fittl.com>
2 siblings, 0 replies; 59+ messages in thread
From: Lukas Fittl @ 2026-04-07 08:05 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Apr 7, 2026 at 1:00 AM Lukas Fittl <lukas@fittl.com> wrote:
>
> Btw, in that same test, when I re-run I get Prefetch but not I/O -- why is that?
>
> Seq Scan on organizations (cost=0.00..7.42 rows=1 width=483) (actual
> time=0.034..0.042 rows=1.00 loops=1)
> Filter: (slug = 'pganalyze'::text)
> Rows Removed by Filter: 113
> Prefetch: avg=1.00 max=1 capacity=94
> Buffers: shared hit=6
>
> (if its all buffers hit, why are there any prefetches at all?)
Here is a more self-contained reproducer:
CREATE TABLE test (id int);
INSERT INTO test SELECT * FROM generate_series(0, 100000);
-- restart server
EXPLAIN (ANALYZE, IO) SELECT * FROM test;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------
Seq Scan on test (cost=0.00..1572.65 rows=112965 width=4) (actual
time=0.075..4.046 rows=100001.00 loops=1)
Prefetch: avg=22.62 max=32 capacity=94
I/O: count=31 waits=5 size=14.29 inprogress=1.77
Buffers: shared read=443
I/O Timings: shared read=0.156
Planning:
Buffers: shared hit=15 read=7
I/O Timings: shared read=0.114
Planning Time: 0.356 ms
Execution Time: 7.020 ms
(10 rows)
EXPLAIN (ANALYZE, IO) SELECT * FROM test;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------
Seq Scan on test (cost=0.00..1572.65 rows=112965 width=4) (actual
time=0.027..5.520 rows=100001.00 loops=1)
Prefetch: avg=1.00 max=1 capacity=94
Buffers: shared hit=443
Planning Time: 0.094 ms
Execution Time: 9.311 ms
(5 rows)
Thanks,
Lukas
--
Lukas Fittl
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
@ 2026-04-07 09:32 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-07 09:32 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/7/26 10:00, Lukas Fittl wrote:
> On Mon, Apr 6, 2026 at 6:14 PM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> On 4/7/26 02:19, Melanie Plageman wrote:
>>> On Mon, Apr 6, 2026 at 4:39 PM Tomas Vondra <tomas@vondra.me> wrote:
>>>>
>>>> On 4/6/26 18:50, Melanie Plageman wrote:
>>>> If you think you can get this pushed, I'll do my best to finalize the
>>>> instrumentation and SeqScan/TidRangeScan parts.
>>>
>>> I've pushed the first patch for index/index-only scans. Attached is
>>> the BHS fix that uses the new pattern. It needs at least one review
>>> before pushing. While I was polishing it, I realized I neglected to
>>> use add_size()/mul_size() in the index-only/index scan patches. So,
>>> 0002 is just a fix commit to do that. Feel free to push these if you
>>> think they're ready. Otherwise, I'll do so pending your review in my
>>> morning.
>>>
>>
>> Thanks!
>>
>> I'll take a look in my morning, and will consider pushing the changes so
>> that I can start pushing the parts adding the new instrumentation.
>
> FWIW, 0001 and 0002 in the v12 patch set also make sense to me from a
> code review just now.
>
>>
>> I've spent a fair amount of time looking at those parts today. Attached
>> is v12 (including the two parts you posted as v11)
>
> 0003 looks good, but commit message needs improvement / justification
> of the change (I assume you had already planned that).
>
Yes, the commit messages need improvements.
> 0004:
>
>> diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
>> index 5b8b521802e..a854c41e963 100644
>> --- a/doc/src/sgml/ref/explain.sgml
>> +++ b/doc/src/sgml/ref/explain.sgml
>> @@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
>> TIMING [ <replaceable class="parameter">boolean</replaceable> ]
>> SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
>> MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
>> + IO [ <replaceable class="parameter">boolean</replaceable> ]
>> FORMAT { TEXT | XML | JSON | YAML }
>> </synopsis>
>> </refsynopsisdiv>
>> @@ -298,6 +299,17 @@ ROLLBACK;
>> </listitem>
>> </varlistentry>
>> + <varlistentry>
>> + <term><literal>IO</literal></term>
>> + <listitem>
>> + <para>
>> + Include information on I/O performed by each node.
>> + This parameter may only be used when <literal>ANALYZE</literal> is also
>> + enabled. It defaults to <literal>FALSE</literal>.
>> + </para>
>> + </listitem>
>> + </varlistentry>
> +
>
> Assuming my understanding of where this works is correct, this needs
> to be qualified I think in terms of which nodes get I/O activity
> shown. Otherwise I as the user will expect e.g. an Index Scan to show
> I/O statistics. Additionally I think this should clearly say "read
> I/O", since writes I/O information wouldn't show for any nodes.
>
Yeah, I was wondering about this too. The "each node" seems to suggest
all nodes get these stats, but then only some scan nodes actually do.
I'm a bit concerned about listing which exact nodes get the stats,
though. AFAICS the explain docs don't do that for any other option.
>> diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
>> index f151f21f9b3..863a9dd0f0d 100644
>> --- a/src/backend/commands/explain.c
>> +++ b/src/backend/commands/explain.c
>> @@ -3984,6 +3990,112 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
>> }
>> }
>> ...
>> + /* prefetch I/O info (only if there were actual I/Os) */
>> + if (stats->io_count > 0)
>> + {
>> + ExplainIndentText(es);
>> + appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
>> + " size=%.2f inprogress=%.2f\n",
>> + stats->io_count, stats->wait_count,
>> + (stats->io_nblocks * 1.0 / stats->io_count),
>> + (stats->io_in_progress * 1.0 / stats->io_count));
>> + }
>
> I've briefly pondered again whether "I/O:" is a bit too broad here in
> EXPLAIN text output, i.e. whether it should qualified it somehow, e.g.
> "I/O Stream", since its just the read stream I/O activity.
>
Hmmm, I don't think we want to mention "Stream" here, that seems more
like an implementation detail to me. Maybe "I/O Reads" or "Read I/O"
would work, though. But OTOH there were discussions about maybe using
the "I/O" both for reads and writes ...
> Right now it looks like this:
>
> Seq Scan on xxx (cost=0.00..7.14 rows=114 width=483) (actual
> time=0.140..0.222 rows=114.00 loops=1)
> Prefetch: avg=1.67 max=3 capacity=94
> I/O: count=3 waits=3 size=2.00 inprogress=1.00
> Buffers: shared read=6
> I/O Timings: shared read=0.158
>
> What if we did this instead:
>
> Seq Scan on xxx (cost=0.00..7.14 rows=114 width=483) (actual
> time=0.140..0.222 rows=114.00 loops=1)
> Prefetch: avg=1.67 max=3 capacity=94
> I/O Stream: count=3 waits=3 size=2.00 inprogress=1.00
> Buffers: shared read=6
> I/O Timings: shared read=0.158
>
> But I'm 50/50 whether that's a big enough problem to debate right now.
> Especially with index prefetching out of the picture this is actually
> less of a problem, since this should only show on scans where the I/O
> activity is mainly (fully?) relying on read streams. If we had index
> prefetching, my understanding is we would have had "I/O" show the read
> stream activity (i.e. table access), but not the index access,
> correct?
>
I think it's OK the way it is. It's possible someone will come up with a
better scheme later, in which case we can adjust this.
> Btw, in that same test, when I re-run I get Prefetch but not I/O -- why is that?
>
> Seq Scan on organizations (cost=0.00..7.42 rows=1 width=483) (actual
> time=0.034..0.042 rows=1.00 loops=1)
> Filter: (slug = 'pganalyze'::text)
> Rows Removed by Filter: 113
> Prefetch: avg=1.00 max=1 capacity=94
> Buffers: shared hit=6
>
> (if its all buffers hit, why are there any prefetches at all?)
>
>> with two or three
>> bigger changes compared to earlier versions:
>>
In the text output this is expected / intentional. The prefetch is based
on number of buffers pulled from the stream - and we use the stream even
with everything in shared buffers. We want to show the prefetch distance
etc. even in this case. But we'll do no I/O in that case, so we don't
show that line.
In the non-text formats we include the fields always.
>> 1) default OFF
>>
>> After thinking about it a bit more, I decided to change the default to
>> OFF. On the one hand I agree it's somewhat similar to BUFFERS, and that
>> option is ON by default now. But on the other hand, we must not clutter
>> EXPLAIN output with too much information, and I'm not convinced IO is
>> worth it. So I changed to OFF by default. That also makes the patches
>> smaller, due to not having to adjust that many tests.
>
> Seems fair in terms of what to do now.
>
OK
>>
>> 2) auto_explain
>>
>> It also occurred to me we should add a matching option to auto_explain,
>> similarly to log_buffers. 0005 does that.
>
> Agreed.
>
OK
>>
>> 3) INSTRUMENT_IO
>>
>> The auto_explain bit also implies we need a new INSTRUMENT_IO constant,
>> to handle this just like BUFFERS. Which also means we can actually
>> collect the stats only when the IO option is specified (instead of for
>> all ANALYZE runs). Which is nice.
>
> FWIW, it'd be the only kind that's not populated by instrument.c (and
> isn't a part of Instrumentation), but I don't see any immediate
> practical issues with having the extra flag.
>
I believe INSTRUMENT_ROWS is not populated there either, so it's not the
only flag not populated there. I've considered adding a need_io field,
but that would be useless / unused.
> It does mean that INSTRUMENT_ALL will turn on INSTRUMENT_IO though for
> the Instrumentation, which will always be a no-op. In practice the
> only users of that are the InstrAlloc calls in
> pg_stat_statements/auto_explain, which I have a patch for to refactor
> anyway (over on the stack-based instrumentation thread, which to be
> clear is PG20 material at this point).
>
> We could probably just drop INSTRUMENT_ALL in the next cycle to avoid
> confusion. For context, that use of INSTRUMENT_ALL won't affect
> es_instrument, just how queryDesc->totaltime is initialized, and what
> it captures at a summary level - and it won't know how to capture I/O
> stats since they are not part of Instrumentation.
>
It is a bit confusing, yes.
> I would suggest noting the fact that its a no-op for Instrumentation
> itself somewhere in code at least, e.g. where INSTRUMENT_IO is defined
> on InstrumentOption.
I'll consider doing that, but I'm not sure which place would be best.
>
> 0005 looks good.
>
> I've skimmed 0006 and 0007 and it makes sense to me why they exist
> (and we should add them), but I haven't reviewed the code for
> correctness.
>
Thanks!
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 12:36 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 15:27 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 2 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-04-07 12:36 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
I've pushed the first two parts earlier today. Here's a v13 with the
remaining pieces rebased on top of that, with updated commit messages
and some additional fixes.
1) There was a bogus else-if in explain.c that would make it impossible
to enable "wal" and "io" at the same time.
2) Replaced one of the "typedef IOStats" with just forward declaration,
to prevent possible conflicting typedefs in the future (even if that
wasn't an issue now).
3) Used (es_instrument & INSTRUMENT_IO) more consistently. A couple
places in the executor still checked just es_instrument, and so would
collect stats even if not needed. Be consistent.
4) Various comment additions / improvements.
I intend to push this later today, after giving it one more round of
review etc.
regards
--
Tomas Vondra
Attachments:
[text/x-patch] v13-0001-Switch-EXPLAIN-to-unaligned-output-for-json-xml-.patch (13.4K, ../../72391376-967e-41b8-806b-41f9efea9a97@vondra.me/2-v13-0001-Switch-EXPLAIN-to-unaligned-output-for-json-xml-.patch)
download | inline diff:
From 1270fb4c87b964021a66ffb261a901e0fdc929af Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v13 1/5] Switch EXPLAIN to unaligned output for json/xml/yaml
Use unaligned output for multiple EXPLAIN queries using non-text format
in regression tests. With aligned output adding/removing explain fields
can be very disruptive, as it often modifies the whole block because of
padding. Unaligned output does not have this issue.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.53.0
[text/x-patch] v13-0002-Add-EXPLAIN-IO-infrastructure-with-BitmapHeapSca.patch (21.9K, ../../72391376-967e-41b8-806b-41f9efea9a97@vondra.me/3-v13-0002-Add-EXPLAIN-IO-infrastructure-with-BitmapHeapSca.patch)
download | inline diff:
From 1d79ca14128b7f1b805bef51880c9603c9c02b8e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:17:22 -0400
Subject: [PATCH v13 2/5] Add EXPLAIN (IO) infrastructure with BitmapHeapScan
support
Allows collecting details about AIO / prefetch for scan nodes backed by
a ReadStream. This may be enabled by a new "IO" option in EXPLAIN, and
it shows information about the prefetch distance and I/O requests.
As of this commit this applies only to BitmapHeapScan, because that's
the only scan node using a ReadStream and collecting instrumentation
from workers in a parallel query. Support for SeqScan and TidRangeScan,
the other scan nodes using ReadStream, will be added in subsequent
commits.
The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (disabled by default). The amount of collected statistics is
very limited, but we don't want to clutter EXPLAIN with too much data.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flag is passed to the
scan, so that the executor and ReadStream has access to it.
The collected stats are designed for ReadStream, but are meant to be
reasonably generic in case a TAM manages I/Os in different ways.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/heap/heapam.c | 10 ++
src/backend/commands/explain.c | 128 ++++++++++++++++++++++
src/backend/commands/explain_state.c | 8 ++
src/backend/executor/nodeBitmapHeapscan.c | 19 +++-
src/backend/storage/aio/read_stream.c | 87 +++++++++++++++
src/include/access/relscan.h | 6 +
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument.h | 1 +
src/include/executor/instrument_node.h | 50 +++++++++
src/include/storage/read_stream.h | 2 +
src/tools/pgindent/typedefs.list | 2 +
13 files changed, 327 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..a854c41e963 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>FALSE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index f6ac5a0897c..89ab9742aa5 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -43,6 +43,7 @@
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/instrument_node.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/lmgr.h"
@@ -1200,6 +1201,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1312,6 +1314,14 @@ heap_beginscan(Relation relation, Snapshot snapshot,
sizeof(TBMIterateResult));
}
+ /* enable read stream instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ {
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ read_stream_enable_stats(scan->rs_read_stream,
+ &scan->rs_base.rs_instrument->io);
+ }
+
scan->rs_vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f151f21f9b3..e9aafdf6e55 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,7 @@
*/
#include "postgres.h"
+#include "access/relscan.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +140,8 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -519,6 +522,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
instrument_option |= INSTRUMENT_BUFFERS;
if (es->wal)
instrument_option |= INSTRUMENT_WAL;
+ if (es->io)
+ instrument_option |= INSTRUMENT_IO;
/*
* We always collect timing for the entire statement, even when node-level
@@ -2008,6 +2013,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -3984,6 +3990,128 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+/*
+ * Print I/O stats - prefetching and I/O performed
+ *
+ * This prints two types of stats - "prefetch" about the prefetching done by
+ * ReadStream, and "I/O" issued by the stream. The prefetch stats are based
+ * on buffers pulled from the stream (even if no I/O is needed). The I/O
+ * information is related to I/O requests issued by the stream.
+ *
+ * The prefetch stats are printed if any buffer was pulled from the stream.
+ * For the I/O stats it depend on the output format. In non-text formats the
+ * information is printed always, while in text format only if there were
+ * any actual I/O requests.
+ */
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print prefetch stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f inprogress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+/*
+ * Show information about prefetch and I/O in a scan node.
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats = {0};
+
+ if (!es->io)
+ return;
+
+ /*
+ * Initialize counters with stats from the local process first.
+ *
+ * The scan descriptor may not exist, e.g. if the scan did not start, or
+ * because of debug_parallel_query=regress. We still want to collect data
+ * from workers.
+ */
+ if (planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument)
+ {
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+ }
+
+ /*
+ * Accumulate data from parallel workers (if any).
+ */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 65dd4111459..0e07a63fca6 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -162,6 +162,8 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ es->io = defGetBoolean(opt);
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -188,6 +190,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d65e2a87b42..83d6478bc2b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -144,13 +144,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -330,6 +337,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect I/O instrumentation for this process */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 0b6cdf7c873..936e08ea450 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -74,6 +74,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -123,6 +124,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -188,6 +192,73 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+/*
+ * Update stream stats about waits for I/O when consuming buffers.
+ *
+ * We count the number of I/O waits while pulling buffers out of a stream.
+ */
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
+/*
+ * Enable collection of stats into the provided IOStats.
+ */
+void
+read_stream_enable_stats(ReadStream *stream, IOStats *stats)
+{
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = stream->max_pinned_buffers;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -426,6 +497,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -1021,6 +1095,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -1036,6 +1111,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
* to avoid having to effectively do another synchronous IO for
* the next block (if it were also a miss).
*/
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -1100,6 +1181,10 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
needed_wait = WaitReadBuffers(&stream->ios[io_index].op);
+ /* Count it as a wait if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
+
Assert(stream->ios_in_progress > 0);
stream->ios_in_progress--;
if (++stream->oldest_io_index == stream->max_ios)
@@ -1228,6 +1313,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fd2076c582a..2ea06a67a63 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,7 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -64,6 +65,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a21c7db5439..c13f05d39db 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -69,6 +69,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 6252fe11f15..97bc7ed49f6 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument.h b/src/include/executor/instrument.h
index cc9fbb0e2f0..f093a52aae0 100644
--- a/src/include/executor/instrument.h
+++ b/src/include/executor/instrument.h
@@ -64,6 +64,7 @@ typedef enum InstrumentOption
INSTRUMENT_BUFFERS = 1 << 1, /* needs buffer usage */
INSTRUMENT_ROWS = 1 << 2, /* needs row count */
INSTRUMENT_WAL = 1 << 3, /* needs WAL usage */
+ INSTRUMENT_IO = 1 << 4, /* needs IO usage */
INSTRUMENT_ALL = PG_INT32_MAX
} InstrumentOption;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index e6a3f9f1941..22a75ccd863 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -48,6 +48,55 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams and I/O
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
@@ -79,6 +128,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..48995c6d534 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+struct IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -103,5 +104,6 @@ extern BlockNumber read_stream_pause(ReadStream *stream);
extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern void read_stream_enable_stats(ReadStream *stream, struct IOStats *stats);
#endif /* READ_STREAM_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9e6a39f5608..98b8d78e693 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1279,6 +1279,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3127,6 +3128,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.53.0
[text/x-patch] v13-0003-auto_explain-Add-new-GUC-auto_explain.log_io.patch (3.6K, ../../72391376-967e-41b8-806b-41f9efea9a97@vondra.me/4-v13-0003-auto_explain-Add-new-GUC-auto_explain.log_io.patch)
download | inline diff:
From 38d3f24ac878be825d80edcc2c90d466791b51b5 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Tue, 7 Apr 2026 02:04:06 +0200
Subject: [PATCH v13 3/5] auto_explain: Add new GUC auto_explain.log_io
Allows enabling the new EXPLAIN "IO" option for auto_explain.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
contrib/auto_explain/auto_explain.c | 15 +++++++++++++++
doc/src/sgml/auto-explain.sgml | 20 ++++++++++++++++++++
2 files changed, 35 insertions(+)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 6ceae1c69ce..9114ced45a7 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -38,6 +38,7 @@ static int auto_explain_log_parameter_max_length = -1; /* bytes or -1 */
static bool auto_explain_log_analyze = false;
static bool auto_explain_log_verbose = false;
static bool auto_explain_log_buffers = false;
+static bool auto_explain_log_io = false;
static bool auto_explain_log_wal = false;
static bool auto_explain_log_triggers = false;
static bool auto_explain_log_timing = true;
@@ -203,6 +204,17 @@ _PG_init(void)
NULL,
NULL);
+ DefineCustomBoolVariable("auto_explain.log_io",
+ "Log I/O statistics.",
+ NULL,
+ &auto_explain_log_io,
+ false,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
DefineCustomBoolVariable("auto_explain.log_wal",
"Log WAL usage.",
NULL,
@@ -343,6 +355,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
queryDesc->instrument_options |= INSTRUMENT_ROWS;
if (auto_explain_log_buffers)
queryDesc->instrument_options |= INSTRUMENT_BUFFERS;
+ if (auto_explain_log_io)
+ queryDesc->instrument_options |= INSTRUMENT_IO;
if (auto_explain_log_wal)
queryDesc->instrument_options |= INSTRUMENT_WAL;
}
@@ -440,6 +454,7 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->analyze = (queryDesc->instrument_options && auto_explain_log_analyze);
es->verbose = auto_explain_log_verbose;
es->buffers = (es->analyze && auto_explain_log_buffers);
+ es->io = (es->analyze && auto_explain_log_io);
es->wal = (es->analyze && auto_explain_log_wal);
es->timing = (es->analyze && auto_explain_log_timing);
es->summary = es->analyze;
diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index ee85a67eb2e..06a8fcc6c5b 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -128,6 +128,26 @@ LOAD 'auto_explain';
</listitem>
</varlistentry>
+ <varlistentry id="auto-explain-configuration-parameters-log-io">
+ <term>
+ <varname>auto_explain.log_io</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>auto_explain.log_io</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <varname>auto_explain.log_io</varname> controls whether I/O usage
+ statistics are printed when an execution plan is logged; it's
+ equivalent to the <literal>IO</literal> option of <command>EXPLAIN</command>.
+ This parameter has no effect
+ unless <varname>auto_explain.log_analyze</varname> is enabled.
+ This parameter is off by default.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="auto-explain-configuration-parameters-log-wal">
<term>
<varname>auto_explain.log_wal</varname> (<type>boolean</type>)
--
2.53.0
[text/x-patch] v13-0004-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch (14.7K, ../../72391376-967e-41b8-806b-41f9efea9a97@vondra.me/5-v13-0004-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch)
download | inline diff:
From 7f0269e4307f5bb8bd850092aba8a5df8411c587 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Tue, 7 Apr 2026 01:38:53 +0200
Subject: [PATCH v13 4/5] Add EXPLAIN (IO) instrumentation for SeqScan
Adds support for EXPLAIN (IO) instrumentation for sequential scans. This
requires adding shared instrumentation for parallel scans, using the
separate DSM approach introduced by dd78e69cfc33.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/backend/commands/explain.c | 25 +++++
src/backend/executor/execParallel.c | 11 +++
src/backend/executor/nodeSeqscan.c | 127 +++++++++++++++++++++++--
src/include/executor/instrument_node.h | 19 ++++
src/include/executor/nodeSeqscan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/test/regress/expected/explain.out | 18 +++-
src/test/regress/sql/explain.sql | 4 +-
src/tools/pgindent/typedefs.list | 2 +
9 files changed, 203 insertions(+), 13 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index e9aafdf6e55..18d424da944 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2032,6 +2032,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -4102,6 +4103,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 1a5ec0c305f..9690f0938ae 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -257,6 +257,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecSeqScanEstimate((SeqScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentEstimate((SeqScanState *) planstate,
+ e->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -500,6 +503,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeDSM((SeqScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitDSM((SeqScanState *) planstate,
+ d->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -1148,6 +1154,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
@@ -1388,6 +1397,8 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
case T_SeqScanState:
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..a4fcf7c7ad4 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -29,6 +29,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
+#include "executor/execParallel.h"
#include "executor/execScan.h"
#include "executor/executor.h"
#include "executor/nodeSeqscan.h"
@@ -65,15 +66,21 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
- 0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ 0, NULL, flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -302,6 +309,22 @@ ExecEndSeqScan(SeqScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
+ /*
+ * Collect I/O stats for this process into shared instrumentation.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (scanDesc && scanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io, &scanDesc->rs_instrument->io);
+ }
+ }
+
/*
* close heap scan
*/
@@ -370,6 +393,13 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -378,9 +408,7 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
}
/* ----------------------------------------------------------------
@@ -410,10 +438,91 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+}
+
+/*
+ * Compute the amount of space we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecSeqScanInstrumentEstimate(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ offsetof(SharedSeqScanInstrumentation, sinstrument) +
+ sizeof(SeqScanInstrumentation) * pcxt->nworkers);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel sequential scan instrumentation.
+ */
+void
+ExecSeqScanInstrumentInitDSM(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedSeqScanInstrumentation *sinstrument;
+ Size size;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument) +
+ sizeof(SeqScanInstrumentation) * pcxt->nworkers;
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->sinstrument = sinstrument;
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.state->es_instrument)
+ return;
+
+ node->sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
+/*
+ * Transfer sequential scan instrumentation from DSM to private memory.
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 22a75ccd863..003dc262b5d 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -266,4 +266,23 @@ typedef struct SharedIncrementalSortInfo
IncrementalSortInfo sinfo[FLEXIBLE_ARRAY_MEMBER];
} SharedIncrementalSortInfo;
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..9c0ad4879d7 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -28,4 +28,13 @@ extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecSeqScanInstrumentEstimate(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitDSM(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
+
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 3ecae7552fc..56febb3204c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1670,6 +1670,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ struct SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..74a4d87801e 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -100,7 +100,7 @@ select explain_filter('explain (buffers, format text) select * from int8_tbl i8'
(1 row)
\a
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
explain_filter
<explain xmlns="http://www.postgresql.org/N/explain">
<Query>
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -149,7 +156,7 @@ explain_filter
</Query>
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
explain_filter
- Plan:
Node Type: "Seq Scan"
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..2f163c64bf6 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -69,8 +69,8 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
\a
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 98b8d78e693..c0436a13ac3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2800,6 +2800,7 @@ SelfJoinCandidate
SemTPadded
SemiAntiJoinFactors
SeqScan
+SeqScanInstrumentation
SeqScanState
SeqTable
SeqTableData
@@ -2864,6 +2865,7 @@ SharedMemoizeInfo
SharedRecordTableEntry
SharedRecordTableKey
SharedRecordTypmodRegistry
+SharedSeqScanInstrumentation
SharedSortInfo
SharedTuplestore
SharedTuplestoreAccessor
--
2.53.0
[text/x-patch] v13-0005-Add-EXPLAIN-IO-instrumentation-for-TidRangeScan.patch (12.3K, ../../72391376-967e-41b8-806b-41f9efea9a97@vondra.me/6-v13-0005-Add-EXPLAIN-IO-instrumentation-for-TidRangeScan.patch)
download | inline diff:
From 5facc503442201929c416bd37233ccd3d85ee45d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:31:18 -0400
Subject: [PATCH v13 5/5] Add EXPLAIN (IO) instrumentation for TidRangeScan
Adds support for EXPLAIN (IO) instrumentation for TidRange scans. This
requires adding shared instrumentation for parallel scans, using the
separate DSM approach introduced by dd78e69cfc33.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/backend/commands/explain.c | 25 +++++
src/backend/executor/execParallel.c | 12 +++
src/backend/executor/nodeTidrangescan.c | 127 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 18 ++++
src/include/executor/nodeTidrangescan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 186 insertions(+), 8 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 18d424da944..b8e4c4eb2da 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2149,6 +2149,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4127,6 +4128,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9690f0938ae..81b87d82fab 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -291,6 +291,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentEstimate((TidRangeScanState *) planstate,
+ e->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -536,6 +539,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitDSM((TidRangeScanState *) planstate,
+ d->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -1157,6 +1163,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
@@ -1430,6 +1439,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitWorker((TidRangeScanState *) planstate,
+ pwcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..550916928cb 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -18,7 +18,9 @@
#include "access/sysattr.h"
#include "access/tableam.h"
#include "catalog/pg_operator.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
+#include "executor/instrument.h"
#include "executor/nodeTidrangescan.h"
#include "nodes/nodeFuncs.h"
#include "utils/rel.h"
@@ -242,12 +244,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +351,20 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /* Collect IO stats for this process into shared instrumentation */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (scan && scan->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io, &scan->rs_instrument->io);
+ }
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -454,6 +477,13 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -462,9 +492,7 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -494,11 +522,94 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+}
+
+/*
+ * Compute the amount of space we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ shm_toc_estimate_chunk(&pcxt->estimator,
+ offsetof(SharedTidRangeScanInstrumentation, sinstrument) +
+ sizeof(TidRangeScanInstrumentation) * pcxt->nworkers);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel scan instrumentation.
+ */
+void
+ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedTidRangeScanInstrumentation *sinstrument;
+ Size size;
+
+ if (!estate->es_instrument || pcxt->nworkers == 0)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument) +
+ sizeof(TidRangeScanInstrumentation) * pcxt->nworkers;
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->trss_sinstrument = sinstrument;
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ if (!node->ss.ps.state->es_instrument)
+ return;
+
+ node->trss_sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ true);
+}
+
+/*
+ * Transfer scan instrumentation from DSM to private memory.
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 003dc262b5d..4076990408e 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -285,4 +285,22 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+
+/*
+ * Instrumentation information for TID range scans
+ */
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..9e7d0a357bb 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -28,4 +28,13 @@ extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelConte
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
+
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 56febb3204c..13359180d25 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1922,6 +1922,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ struct SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c0436a13ac3..0cd41106975 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2867,6 +2867,7 @@ SharedRecordTableKey
SharedRecordTypmodRegistry
SharedSeqScanInstrumentation
SharedSortInfo
+SharedTidRangeScanInstrumentation
SharedTuplestore
SharedTuplestoreAccessor
SharedTuplestoreChunk
@@ -3171,6 +3172,7 @@ TidOpExpr
TidPath
TidRangePath
TidRangeScan
+TidRangeScanInstrumentation
TidRangeScanState
TidScan
TidScanState
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 15:27 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 16:13 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
1 sibling, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-07 15:27 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Apr 7, 2026 at 8:36 AM Tomas Vondra <tomas@vondra.me> wrote:
>
> 3) Used (es_instrument & INSTRUMENT_IO) more consistently. A couple
> places in the executor still checked just es_instrument, and so would
> collect stats even if not needed. Be consistent.
I would argue for only allocating the shared instrumentation if they
will be displayed, but I don't feel that strongly about it.
I took another look at just 0002 just to double-check the read stream
counters. It seems fine.
Perhaps we should free the TableScanInstrumentation in heap_endscan().
It isn't that important of a leak, but the other things palloc'd in
heap_beginscan() are freed.
I also wondered if it is clear without a comment why we don't count a
wait when READ_BUFFERS_SYNCHRONOUSLY sets needed_wait (which we added
to make sure distance ramps up). I think it's fine; I'm just musing.
- Melanie
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 15:27 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-07 16:13 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 18:12 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-07 16:13 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Lukas Fittl <lukas@fittl.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/7/26 17:27, Melanie Plageman wrote:
> On Tue, Apr 7, 2026 at 8:36 AM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> 3) Used (es_instrument & INSTRUMENT_IO) more consistently. A couple
>> places in the executor still checked just es_instrument, and so would
>> collect stats even if not needed. Be consistent.
>
> I would argue for only allocating the shared instrumentation if they
> will be displayed, but I don't feel that strongly about it.
>
OK
> I took another look at just 0002 just to double-check the read stream
> counters. It seems fine.
> Perhaps we should free the TableScanInstrumentation in heap_endscan().
> It isn't that important of a leak, but the other things palloc'd in
> heap_beginscan() are freed.
>
Good point, will fix.
> I also wondered if it is clear without a comment why we don't count a
> wait when READ_BUFFERS_SYNCHRONOUSLY sets needed_wait (which we added
> to make sure distance ramps up). I think it's fine; I'm just musing.
>
I believe this is actually a bug, the read_stream_count_wait() should be
after the block checking for READ_BUFFERS_SYNCHRONOUSLY. I recall Andres
mentioned this in one of this reviews, but I either forgot to address
that, or it got lost then transferring this between this thread and the
index prefetching one.
Thanks!
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 15:27 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 16:13 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 18:12 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 59+ messages in thread
From: Andres Freund @ 2026-04-07 18:12 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-04-07 18:13:18 +0200, Tomas Vondra wrote:
> On 4/7/26 17:27, Melanie Plageman wrote:
> > I took another look at just 0002 just to double-check the read stream
> > counters. It seems fine.
> > Perhaps we should free the TableScanInstrumentation in heap_endscan().
> > It isn't that important of a leak, but the other things palloc'd in
> > heap_beginscan() are freed.
> >
>
> Good point, will fix.
>
> > I also wondered if it is clear without a comment why we don't count a
> > wait when READ_BUFFERS_SYNCHRONOUSLY sets needed_wait (which we added
> > to make sure distance ramps up). I think it's fine; I'm just musing.
> >
>
> I believe this is actually a bug, the read_stream_count_wait() should be
> after the block checking for READ_BUFFERS_SYNCHRONOUSLY. I recall Andres
> mentioned this in one of this reviews, but I either forgot to address
> that, or it got lost then transferring this between this thread and the
> index prefetching one.
Yea, I think it's actually somewhat important that those will be counted as
waits, since they *are* blocking reads. It's just that the blocking read
happends during the start of the IO, so we don't have to wait in
WaitReadBuffers(). You'd otherwise severely undercounting reads in cases where
the distance ramps up and down a lot
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 16:07 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
1 sibling, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-07 16:07 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Apr 7, 2026 at 8:36 AM Tomas Vondra <tomas@vondra.me> wrote:
>
> I've pushed the first two parts earlier today. Here's a v13 with the
> remaining pieces rebased on top of that, with updated commit messages
> and some additional fixes.
I noticed a couple of things while reviewing 0004.
I see I had bitmapheapscan use
if (!node->ss.ps.instrument || pcxt->nworkers == 0)
return;
while seq scan uses
if (!estate->es_instrument || pcxt->nworkers == 0)
return;
That does seem worth being consistent about. Though the estate one is
probably better to use and changing bitmapheapscan in this commit
might be noisy... I don't feel strongly either way.
In ExecSeqScanInstrumentInitWorker() -> shm_toc_lookup() noerror is true:
node->sinstrument = shm_toc_lookup(pwcxt->toc,
node->ss.ps.plan->plan_node_id
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
true);
I went back and forth on this for bitmapheapscan and ended up going
with the guard
if (!node->ss.ps.instrument)
return;
and passing noerror as false. I thought it would be better to get an
error if something went wrong.
If you keep noerror true, there's no reason to check for es_instrument.
I also noticed that seq scan doesn't use add_size/mul_size for the
size calculations when estimating and allocating DSM. I switched to
using them in bitmapheapscan because I realized other code in that
file used them. Seems seq scan was doing plain arithmetic without
those helpers already. And overflow is unlikely here. But I wonder if
it is worth using them?
It doesn't matter for the Retrieve functions because overflow would
have caused us to error out at estimation or allocation time.
Most of the seq scan specific stuff above applies to tid range scan as well.
One other point about 0002: I wonder if the text explain output should
say "in-progress" instead of "inprogress"
- Melanie
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-07 17:34 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-07 17:34 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Lukas Fittl <lukas@fittl.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/7/26 18:07, Melanie Plageman wrote:
> On Tue, Apr 7, 2026 at 8:36 AM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> I've pushed the first two parts earlier today. Here's a v13 with the
>> remaining pieces rebased on top of that, with updated commit messages
>> and some additional fixes.
>
> I noticed a couple of things while reviewing 0004.
>
> I see I had bitmapheapscan use
> if (!node->ss.ps.instrument || pcxt->nworkers == 0)
> return;
>
> while seq scan uses
> if (!estate->es_instrument || pcxt->nworkers == 0)
> return;
>
> That does seem worth being consistent about. Though the estate one is
> probably better to use and changing bitmapheapscan in this commit
> might be noisy... I don't feel strongly either way.
>
I'm not sure this is just a question of consistency, because BHS may
need the shared instrumentation even if (es_instrument = 0), no? While
the seqscan/tidrange scan only need it with EXPLAIN(IO).
So I think the two nodes should check
((estate->es_instrument & INSTRUMENT_IO) == 0)
The attached v14 does it this way. I left BHS to check ps.instrument, I
think that's fine.
> In ExecSeqScanInstrumentInitWorker() -> shm_toc_lookup() noerror is true:
>
> node->sinstrument = shm_toc_lookup(pwcxt->toc,
> node->ss.ps.plan->plan_node_id
> + PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
> true);
>
> I went back and forth on this for bitmapheapscan and ended up going
> with the guard
> if (!node->ss.ps.instrument)
> return;
> and passing noerror as false. I thought it would be better to get an
> error if something went wrong.
>
> If you keep noerror true, there's no reason to check for es_instrument.
I agree this should use noerror=false. It's an error to not find the
expected DSM chunk, and we don't want to hide that. It'd probably fail
with a segfault later, but that's hardly a better outcome.
>
> I also noticed that seq scan doesn't use add_size/mul_size for the
> size calculations when estimating and allocating DSM. I switched to
> using them in bitmapheapscan because I realized other code in that
> file used them. Seems seq scan was doing plain arithmetic without
> those helpers already. And overflow is unlikely here. But I wonder if
> it is worth using them?
>
Seems better to use that, so done in v14.
> It doesn't matter for the Retrieve functions because overflow would
> have caused us to error out at estimation or allocation time.
>
> Most of the seq scan specific stuff above applies to tid range scan as well.
>
OK, fixed that too.
> One other point about 0002: I wonder if the text explain output should
> say "in-progress" instead of "inprogress"
>
True. Changed.
Thanks!
--
Tomas Vondra
Attachments:
[text/x-patch] v14-0001-Switch-EXPLAIN-to-unaligned-output-for-json-xml-.patch (13.4K, ../../552ecf6c-e8a2-4183-afc4-eff592674475@vondra.me/2-v14-0001-Switch-EXPLAIN-to-unaligned-output-for-json-xml-.patch)
download | inline diff:
From 1270fb4c87b964021a66ffb261a901e0fdc929af Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas@vondra.me>
Date: Tue, 31 Mar 2026 13:44:23 +0200
Subject: [PATCH v14 1/5] Switch EXPLAIN to unaligned output for json/xml/yaml
Use unaligned output for multiple EXPLAIN queries using non-text format
in regression tests. With aligned output adding/removing explain fields
can be very disruptive, as it often modifies the whole block because of
padding. Unaligned output does not have this issue.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/test/regress/expected/explain.out | 296 +++++++++++++-------------
src/test/regress/sql/explain.sql | 5 +-
2 files changed, 150 insertions(+), 151 deletions(-)
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index 7c1f26b182c..dc31c7ce9f9 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -93,164 +93,160 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
Execution Time: N.N ms
(3 rows)
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
- explain_filter
---------------------------------------------------------
- <explain xmlns="http://www.postgresql.org/N/explain"> +
- <Query> +
- <Plan> +
- <Node-Type>Seq Scan</Node-Type> +
- <Parallel-Aware>false</Parallel-Aware> +
- <Async-Capable>false</Async-Capable> +
- <Relation-Name>int8_tbl</Relation-Name> +
- <Alias>i8</Alias> +
- <Startup-Cost>N.N</Startup-Cost> +
- <Total-Cost>N.N</Total-Cost> +
- <Plan-Rows>N</Plan-Rows> +
- <Plan-Width>N</Plan-Width> +
- <Actual-Startup-Time>N.N</Actual-Startup-Time> +
- <Actual-Total-Time>N.N</Actual-Total-Time> +
- <Actual-Rows>N.N</Actual-Rows> +
- <Actual-Loops>N</Actual-Loops> +
- <Disabled>false</Disabled> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Plan> +
- <Planning> +
- <Shared-Hit-Blocks>N</Shared-Hit-Blocks> +
- <Shared-Read-Blocks>N</Shared-Read-Blocks> +
- <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>+
- <Shared-Written-Blocks>N</Shared-Written-Blocks>+
- <Local-Hit-Blocks>N</Local-Hit-Blocks> +
- <Local-Read-Blocks>N</Local-Read-Blocks> +
- <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks> +
- <Local-Written-Blocks>N</Local-Written-Blocks> +
- <Temp-Read-Blocks>N</Temp-Read-Blocks> +
- <Temp-Written-Blocks>N</Temp-Written-Blocks> +
- </Planning> +
- <Planning-Time>N.N</Planning-Time> +
- <Triggers> +
- </Triggers> +
- <Execution-Time>N.N</Execution-Time> +
- </Query> +
- </explain>
-(1 row)
-
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
- explain_filter
--------------------------------
- - Plan: +
- Node Type: "Seq Scan" +
- Parallel Aware: false +
- Async Capable: false +
- Relation Name: "int8_tbl"+
- Alias: "i8" +
- Startup Cost: N.N +
- Total Cost: N.N +
- Plan Rows: N +
- Plan Width: N +
- Actual Startup Time: N.N +
- Actual Total Time: N.N +
- Actual Rows: N.N +
- Actual Loops: N +
- Disabled: false +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning: +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Planning Time: N.N +
- Triggers: +
- Serialization: +
- Time: N.N +
- Output Volume: N +
- Format: "text" +
- Shared Hit Blocks: N +
- Shared Read Blocks: N +
- Shared Dirtied Blocks: N +
- Shared Written Blocks: N +
- Local Hit Blocks: N +
- Local Read Blocks: N +
- Local Dirtied Blocks: N +
- Local Written Blocks: N +
- Temp Read Blocks: N +
- Temp Written Blocks: N +
- Execution Time: N.N
-(1 row)
-
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
explain_filter
---------------------------------------------------------
Seq Scan on int8_tbl i8 (cost=N.N..N.N rows=N width=N)
(1 row)
+\a
+select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+explain_filter
+<explain xmlns="http://www.postgresql.org/N/explain">
+ <Query>
+ <Plan>
+ <Node-Type>Seq Scan</Node-Type>
+ <Parallel-Aware>false</Parallel-Aware>
+ <Async-Capable>false</Async-Capable>
+ <Relation-Name>int8_tbl</Relation-Name>
+ <Alias>i8</Alias>
+ <Startup-Cost>N.N</Startup-Cost>
+ <Total-Cost>N.N</Total-Cost>
+ <Plan-Rows>N</Plan-Rows>
+ <Plan-Width>N</Plan-Width>
+ <Actual-Startup-Time>N.N</Actual-Startup-Time>
+ <Actual-Total-Time>N.N</Actual-Total-Time>
+ <Actual-Rows>N.N</Actual-Rows>
+ <Actual-Loops>N</Actual-Loops>
+ <Disabled>false</Disabled>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Plan>
+ <Planning>
+ <Shared-Hit-Blocks>N</Shared-Hit-Blocks>
+ <Shared-Read-Blocks>N</Shared-Read-Blocks>
+ <Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
+ <Shared-Written-Blocks>N</Shared-Written-Blocks>
+ <Local-Hit-Blocks>N</Local-Hit-Blocks>
+ <Local-Read-Blocks>N</Local-Read-Blocks>
+ <Local-Dirtied-Blocks>N</Local-Dirtied-Blocks>
+ <Local-Written-Blocks>N</Local-Written-Blocks>
+ <Temp-Read-Blocks>N</Temp-Read-Blocks>
+ <Temp-Written-Blocks>N</Temp-Written-Blocks>
+ </Planning>
+ <Planning-Time>N.N</Planning-Time>
+ <Triggers>
+ </Triggers>
+ <Execution-Time>N.N</Execution-Time>
+ </Query>
+</explain>
+(1 row)
+select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+explain_filter
+- Plan:
+ Node Type: "Seq Scan"
+ Parallel Aware: false
+ Async Capable: false
+ Relation Name: "int8_tbl"
+ Alias: "i8"
+ Startup Cost: N.N
+ Total Cost: N.N
+ Plan Rows: N
+ Plan Width: N
+ Actual Startup Time: N.N
+ Actual Total Time: N.N
+ Actual Rows: N.N
+ Actual Loops: N
+ Disabled: false
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning:
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Planning Time: N.N
+ Triggers:
+ Serialization:
+ Time: N.N
+ Output Volume: N
+ Format: "text"
+ Shared Hit Blocks: N
+ Shared Read Blocks: N
+ Shared Dirtied Blocks: N
+ Shared Written Blocks: N
+ Local Hit Blocks: N
+ Local Read Blocks: N
+ Local Dirtied Blocks: N
+ Local Written Blocks: N
+ Temp Read Blocks: N
+ Temp Written Blocks: N
+ Execution Time: N.N
+(1 row)
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
- explain_filter
-------------------------------------
- [ +
- { +
- "Plan": { +
- "Node Type": "Seq Scan", +
- "Parallel Aware": false, +
- "Async Capable": false, +
- "Relation Name": "int8_tbl",+
- "Alias": "i8", +
- "Startup Cost": N.N, +
- "Total Cost": N.N, +
- "Plan Rows": N, +
- "Plan Width": N, +
- "Disabled": false, +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- }, +
- "Planning": { +
- "Shared Hit Blocks": N, +
- "Shared Read Blocks": N, +
- "Shared Dirtied Blocks": N, +
- "Shared Written Blocks": N, +
- "Local Hit Blocks": N, +
- "Local Read Blocks": N, +
- "Local Dirtied Blocks": N, +
- "Local Written Blocks": N, +
- "Temp Read Blocks": N, +
- "Temp Written Blocks": N +
- } +
- } +
- ]
+explain_filter
+[
+ {
+ "Plan": {
+ "Node Type": "Seq Scan",
+ "Parallel Aware": false,
+ "Async Capable": false,
+ "Relation Name": "int8_tbl",
+ "Alias": "i8",
+ "Startup Cost": N.N,
+ "Total Cost": N.N,
+ "Plan Rows": N,
+ "Plan Width": N,
+ "Disabled": false,
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ },
+ "Planning": {
+ "Shared Hit Blocks": N,
+ "Shared Read Blocks": N,
+ "Shared Dirtied Blocks": N,
+ "Shared Written Blocks": N,
+ "Local Hit Blocks": N,
+ "Local Read Blocks": N,
+ "Local Dirtied Blocks": N,
+ "Local Written Blocks": N,
+ "Temp Read Blocks": N,
+ "Temp Written Blocks": N
+ }
+ }
+]
(1 row)
-
+\a
-- Check expansion of window definitions
select explain_filter('explain verbose select sum(unique1) over w, sum(unique2) over (w order by hundred), sum(tenthous) over (w order by hundred) from tenk1 window w as (partition by ten)');
explain_filter
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index ebdab42604b..8f10e1aff55 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -66,10 +66,13 @@ select explain_filter('explain select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers off, verbose) select * from int8_tbl i8');
select explain_filter('explain (analyze, buffers, format text) select * from int8_tbl i8');
+select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
+
+\a
select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
-select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
+\a
-- Check expansion of window definitions
--
2.53.0
[text/x-patch] v14-0002-Add-EXPLAIN-IO-infrastructure-with-BitmapHeapSca.patch (22.2K, ../../552ecf6c-e8a2-4183-afc4-eff592674475@vondra.me/3-v14-0002-Add-EXPLAIN-IO-infrastructure-with-BitmapHeapSca.patch)
download | inline diff:
From 74e88f08d87801799d79c97d451da3b89ee57de9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Sun, 5 Apr 2026 17:17:22 -0400
Subject: [PATCH v14 2/5] Add EXPLAIN (IO) infrastructure with BitmapHeapScan
support
Allows collecting details about AIO / prefetch for scan nodes backed by
a ReadStream. This may be enabled by a new "IO" option in EXPLAIN, and
it shows information about the prefetch distance and I/O requests.
As of this commit this applies only to BitmapHeapScan, because that's
the only scan node using a ReadStream and collecting instrumentation
from workers in a parallel query. Support for SeqScan and TidRangeScan,
the other scan nodes using ReadStream, will be added in subsequent
commits.
The stats are collected only when required by EXPLAIN ANALYZE, with the
IO option (disabled by default). The amount of collected statistics is
very limited, but we don't want to clutter EXPLAIN with too much data.
The IOStats struct is stored in the scan descriptor as a field, next to
other fields used by table AMs. A pointer to the field is passed to the
ReadStream, and updated directly.
It's the responsibility of the table AM to allocate the struct (e.g. in
ambeginscan) whenever the flag SO_SCAN_INSTRUMENT flag is passed to the
scan, so that the executor and ReadStream has access to it.
The collected stats are designed for ReadStream, but are meant to be
reasonably generic in case a TAM manages I/Os in different ways.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
doc/src/sgml/ref/explain.sgml | 12 ++
src/backend/access/heap/heapam.c | 13 +++
src/backend/commands/explain.c | 128 ++++++++++++++++++++++
src/backend/commands/explain_state.c | 8 ++
src/backend/executor/nodeBitmapHeapscan.c | 19 +++-
src/backend/storage/aio/read_stream.c | 87 +++++++++++++++
src/include/access/relscan.h | 6 +
src/include/access/tableam.h | 3 +
src/include/commands/explain_state.h | 1 +
src/include/executor/instrument.h | 1 +
src/include/executor/instrument_node.h | 50 +++++++++
src/include/storage/read_stream.h | 2 +
src/tools/pgindent/typedefs.list | 2 +
13 files changed, 330 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index 5b8b521802e..a854c41e963 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -46,6 +46,7 @@ EXPLAIN [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] <rep
TIMING [ <replaceable class="parameter">boolean</replaceable> ]
SUMMARY [ <replaceable class="parameter">boolean</replaceable> ]
MEMORY [ <replaceable class="parameter">boolean</replaceable> ]
+ IO [ <replaceable class="parameter">boolean</replaceable> ]
FORMAT { TEXT | XML | JSON | YAML }
</synopsis>
</refsynopsisdiv>
@@ -298,6 +299,17 @@ ROLLBACK;
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>IO</literal></term>
+ <listitem>
+ <para>
+ Include information on I/O performed by each node.
+ This parameter may only be used when <literal>ANALYZE</literal> is also
+ enabled. It defaults to <literal>FALSE</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>FORMAT</literal></term>
<listitem>
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index f6ac5a0897c..adf716eca54 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -43,6 +43,7 @@
#include "catalog/pg_database.h"
#include "catalog/pg_database_d.h"
#include "commands/vacuum.h"
+#include "executor/instrument_node.h"
#include "pgstat.h"
#include "port/pg_bitutils.h"
#include "storage/lmgr.h"
@@ -1200,6 +1201,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_nkeys = nkeys;
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
+ scan->rs_base.rs_instrument = NULL;
scan->rs_strategy = NULL; /* set in initscan */
scan->rs_cbuf = InvalidBuffer;
@@ -1312,6 +1314,14 @@ heap_beginscan(Relation relation, Snapshot snapshot,
sizeof(TBMIterateResult));
}
+ /* enable read stream instrumentation */
+ if (flags & SO_SCAN_INSTRUMENT)
+ {
+ scan->rs_base.rs_instrument = palloc0_object(TableScanInstrumentation);
+ read_stream_enable_stats(scan->rs_read_stream,
+ &scan->rs_base.rs_instrument->io);
+ }
+
scan->rs_vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
@@ -1415,6 +1425,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_instrument)
+ pfree(scan->rs_base.rs_instrument);
+
pfree(scan);
}
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index f151f21f9b3..ca3071f4ca2 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -13,6 +13,7 @@
*/
#include "postgres.h"
+#include "access/relscan.h"
#include "access/xact.h"
#include "catalog/pg_type.h"
#include "commands/createas.h"
@@ -139,6 +140,8 @@ static void show_hashagg_info(AggState *aggstate, ExplainState *es);
static void show_indexsearches_info(PlanState *planstate, ExplainState *es);
static void show_tidbitmap_info(BitmapHeapScanState *planstate,
ExplainState *es);
+static void show_scan_io_usage(ScanState *planstate,
+ ExplainState *es);
static void show_instrumentation_count(const char *qlabel, int which,
PlanState *planstate, ExplainState *es);
static void show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es);
@@ -519,6 +522,8 @@ ExplainOnePlan(PlannedStmt *plannedstmt, IntoClause *into, ExplainState *es,
instrument_option |= INSTRUMENT_BUFFERS;
if (es->wal)
instrument_option |= INSTRUMENT_WAL;
+ if (es->io)
+ instrument_option |= INSTRUMENT_IO;
/*
* We always collect timing for the entire statement, even when node-level
@@ -2008,6 +2013,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
show_tidbitmap_info((BitmapHeapScanState *) planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_SampleScan:
show_tablesample(((SampleScan *) plan)->tablesample,
@@ -3984,6 +3990,128 @@ show_tidbitmap_info(BitmapHeapScanState *planstate, ExplainState *es)
}
}
+/*
+ * Print I/O stats - prefetching and I/O performed
+ *
+ * This prints two types of stats - "prefetch" about the prefetching done by
+ * ReadStream, and "I/O" issued by the stream. The prefetch stats are based
+ * on buffers pulled from the stream (even if no I/O is needed). The I/O
+ * information is related to I/O requests issued by the stream.
+ *
+ * The prefetch stats are printed if any buffer was pulled from the stream.
+ * For the I/O stats it depend on the output format. In non-text formats the
+ * information is printed always, while in text format only if there were
+ * any actual I/O requests.
+ */
+static void
+print_io_usage(ExplainState *es, IOStats *stats)
+{
+ /* don't print prefetch stats if there's nothing to report */
+ if (stats->prefetch_count > 0)
+ {
+ if (es->format == EXPLAIN_FORMAT_TEXT)
+ {
+ /* prefetch distance info */
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
+ (stats->distance_sum * 1.0 / stats->prefetch_count),
+ stats->distance_max,
+ stats->distance_capacity);
+
+ /* prefetch I/O info (only if there were actual I/Os) */
+ if (stats->io_count > 0)
+ {
+ ExplainIndentText(es);
+ appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
+ " size=%.2f in-progress=%.2f\n",
+ stats->io_count, stats->wait_count,
+ (stats->io_nblocks * 1.0 / stats->io_count),
+ (stats->io_in_progress * 1.0 / stats->io_count));
+ }
+ }
+ else
+ {
+ ExplainPropertyFloat("Average Prefetch Distance", NULL,
+ (stats->distance_sum * 1.0 / stats->prefetch_count), 3, es);
+ ExplainPropertyInteger("Max Prefetch Distance", NULL,
+ stats->distance_max, es);
+ ExplainPropertyInteger("Prefetch Capacity", NULL,
+ stats->distance_capacity, es);
+
+ ExplainPropertyUInteger("I/O Count", NULL,
+ stats->io_count, es);
+ ExplainPropertyUInteger("I/O Waits", NULL,
+ stats->wait_count, es);
+ ExplainPropertyFloat("Average I/O Size", NULL,
+ (stats->io_nblocks * 1.0 / Max(1, stats->io_count)), 3, es);
+ ExplainPropertyFloat("Average I/Os In Progress", NULL,
+ (stats->io_in_progress * 1.0 / Max(1, stats->io_count)), 3, es);
+ }
+ }
+}
+
+/*
+ * Show information about prefetch and I/O in a scan node.
+ */
+static void
+show_scan_io_usage(ScanState *planstate, ExplainState *es)
+{
+ Plan *plan = planstate->ps.plan;
+ IOStats stats = {0};
+
+ if (!es->io)
+ return;
+
+ /*
+ * Initialize counters with stats from the local process first.
+ *
+ * The scan descriptor may not exist, e.g. if the scan did not start, or
+ * because of debug_parallel_query=regress. We still want to collect data
+ * from workers.
+ */
+ if (planstate->ss_currentScanDesc &&
+ planstate->ss_currentScanDesc->rs_instrument)
+ {
+ stats = planstate->ss_currentScanDesc->rs_instrument->io;
+ }
+
+ /*
+ * Accumulate data from parallel workers (if any).
+ */
+ switch (nodeTag(plan))
+ {
+ case T_BitmapHeapScan:
+ {
+ SharedBitmapHeapInstrumentation *sinstrument
+ = ((BitmapHeapScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ BitmapHeapScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
+ break;
+ }
+ default:
+ /* ignore other plans */
+ return;
+ }
+
+ print_io_usage(es, &stats);
+}
+
/*
* If it's EXPLAIN ANALYZE, show instrumentation information for a plan node
*
diff --git a/src/backend/commands/explain_state.c b/src/backend/commands/explain_state.c
index 65dd4111459..0e07a63fca6 100644
--- a/src/backend/commands/explain_state.c
+++ b/src/backend/commands/explain_state.c
@@ -162,6 +162,8 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
"EXPLAIN", opt->defname, p),
parser_errposition(pstate, opt->location)));
}
+ else if (strcmp(opt->defname, "io") == 0)
+ es->io = defGetBoolean(opt);
else if (!ApplyExtensionExplainOption(es, opt, pstate))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -188,6 +190,12 @@ ParseExplainOptionList(ExplainState *es, List *options, ParseState *pstate)
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("EXPLAIN option %s requires ANALYZE", "TIMING")));
+ /* check that IO is used with EXPLAIN ANALYZE */
+ if (es->io && !es->analyze)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("EXPLAIN option %s requires ANALYZE", "IO")));
+
/* check that serialize is used with EXPLAIN ANALYZE */
if (es->serialize != EXPLAIN_SERIALIZE_NONE && !es->analyze)
ereport(ERROR,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d65e2a87b42..83d6478bc2b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -144,13 +144,20 @@ BitmapTableScanSetup(BitmapHeapScanState *node)
*/
if (!node->ss.ss_currentScanDesc)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
node->ss.ss_currentScanDesc =
table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
}
node->ss.ss_currentScanDesc->st.rs_tbmiterator = tbmiterator;
@@ -330,6 +337,14 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
si->exact_pages += node->stats.exact_pages;
si->lossy_pages += node->stats.lossy_pages;
+
+ /* collect I/O instrumentation for this process */
+ if (node->ss.ss_currentScanDesc &&
+ node->ss.ss_currentScanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io,
+ &node->ss.ss_currentScanDesc->rs_instrument->io);
+ }
}
/*
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index 0b6cdf7c873..b6fce4e7cc6 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -74,6 +74,7 @@
#include "postgres.h"
#include "miscadmin.h"
+#include "executor/instrument_node.h"
#include "storage/aio.h"
#include "storage/fd.h"
#include "storage/smgr.h"
@@ -123,6 +124,9 @@ struct ReadStream
bool advice_enabled;
bool temporary;
+ /* scan stats counters */
+ IOStats *stats;
+
/*
* One-block buffer to support 'ungetting' a block number, to resolve flow
* control problems when I/Os are split.
@@ -188,6 +192,73 @@ block_range_read_stream_cb(ReadStream *stream,
return InvalidBlockNumber;
}
+/*
+ * Update stream stats with current pinned buffer depth.
+ *
+ * Called once per buffer returned to the consumer in read_stream_next_buffer().
+ * Records the number of pinned buffers at that moment, so we can compute the
+ * average look-ahead depth.
+ */
+static inline void
+read_stream_count_prefetch(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->prefetch_count++;
+ stats->distance_sum += stream->pinned_buffers;
+ if (stream->pinned_buffers > stats->distance_max)
+ stats->distance_max = stream->pinned_buffers;
+}
+
+/*
+ * Update stream stats about size of I/O requests.
+ *
+ * We count the number of I/O requests, size of requests (counted in blocks)
+ * and number of in-progress I/Os.
+ */
+static inline void
+read_stream_count_io(ReadStream *stream, int nblocks, int in_progress)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->io_count++;
+ stats->io_nblocks += nblocks;
+ stats->io_in_progress += in_progress;
+}
+
+/*
+ * Update stream stats about waits for I/O when consuming buffers.
+ *
+ * We count the number of I/O waits while pulling buffers out of a stream.
+ */
+static inline void
+read_stream_count_wait(ReadStream *stream)
+{
+ IOStats *stats = stream->stats;
+
+ if (stats == NULL)
+ return;
+
+ stats->wait_count++;
+}
+
+/*
+ * Enable collection of stats into the provided IOStats.
+ */
+void
+read_stream_enable_stats(ReadStream *stream, IOStats *stats)
+{
+ stream->stats = stats;
+ if (stream->stats)
+ stream->stats->distance_capacity = stream->max_pinned_buffers;
+}
+
/*
* Ask the callback which block it would like us to read next, with a one block
* buffer in front to allow read_stream_unget_block() to work.
@@ -426,6 +497,9 @@ read_stream_start_pending_read(ReadStream *stream)
Assert(stream->ios_in_progress < stream->max_ios);
stream->ios_in_progress++;
stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+
+ /* update I/O stats */
+ read_stream_count_io(stream, nblocks, stream->ios_in_progress);
}
/*
@@ -1021,6 +1095,7 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
flags)))
{
/* Fast return. */
+ read_stream_count_prefetch(stream);
return buffer;
}
@@ -1036,6 +1111,12 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
* to avoid having to effectively do another synchronous IO for
* the next block (if it were also a miss).
*/
+
+ /* update I/O stats */
+ read_stream_count_io(stream, 1, stream->ios_in_progress);
+
+ /* update prefetch distance */
+ read_stream_count_prefetch(stream);
}
else
{
@@ -1116,6 +1197,10 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
if (stream->ios[io_index].op.flags & READ_BUFFERS_SYNCHRONOUSLY)
needed_wait = true;
+ /* Count it as a wait if we need to wait for IO */
+ if (needed_wait)
+ read_stream_count_wait(stream);
+
/*
* Have the read-ahead distance ramp up rapidly after we needed to
* wait for IO. We only increase the read-ahead-distance when we
@@ -1228,6 +1313,8 @@ read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
}
#endif
+ read_stream_count_prefetch(stream);
+
/* Pin transferred to caller. */
Assert(stream->pinned_buffers > 0);
stream->pinned_buffers--;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fd2076c582a..2ea06a67a63 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,7 @@
struct ParallelTableScanDescData;
+struct TableScanInstrumentation;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -64,6 +65,11 @@ typedef struct TableScanDescData
struct ParallelTableScanDescData *rs_parallel; /* parallel scan
* information */
+
+ /*
+ * Instrumentation counters maintained by all table AMs.
+ */
+ struct TableScanInstrumentation *rs_instrument;
} TableScanDescData;
typedef struct TableScanDescData *TableScanDesc;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a21c7db5439..c13f05d39db 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -69,6 +69,9 @@ typedef enum ScanOptions
/* set if the query doesn't modify the relation */
SO_HINT_REL_READ_ONLY = 1 << 10,
+
+ /* collect scan instrumentation */
+ SO_SCAN_INSTRUMENT = 1 << 11,
} ScanOptions;
/*
diff --git a/src/include/commands/explain_state.h b/src/include/commands/explain_state.h
index 6252fe11f15..97bc7ed49f6 100644
--- a/src/include/commands/explain_state.h
+++ b/src/include/commands/explain_state.h
@@ -55,6 +55,7 @@ typedef struct ExplainState
bool summary; /* print total planning and execution timing */
bool memory; /* print planner's memory usage information */
bool settings; /* print modified settings */
+ bool io; /* print info about IO (prefetch, ...) */
bool generic; /* generate a generic plan */
ExplainSerializeOption serialize; /* serialize the query's output? */
ExplainFormat format; /* output format */
diff --git a/src/include/executor/instrument.h b/src/include/executor/instrument.h
index cc9fbb0e2f0..f093a52aae0 100644
--- a/src/include/executor/instrument.h
+++ b/src/include/executor/instrument.h
@@ -64,6 +64,7 @@ typedef enum InstrumentOption
INSTRUMENT_BUFFERS = 1 << 1, /* needs buffer usage */
INSTRUMENT_ROWS = 1 << 2, /* needs row count */
INSTRUMENT_WAL = 1 << 3, /* needs WAL usage */
+ INSTRUMENT_IO = 1 << 4, /* needs IO usage */
INSTRUMENT_ALL = PG_INT32_MAX
} InstrumentOption;
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index e6a3f9f1941..22a75ccd863 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -48,6 +48,55 @@ typedef struct SharedAggInfo
} SharedAggInfo;
+/* ---------------------
+ * Instrumentation information about read streams and I/O
+ * ---------------------
+ */
+typedef struct IOStats
+{
+ /* number of buffers returned to consumer (for averaging distance) */
+ uint64 prefetch_count;
+
+ /* sum of pinned_buffers sampled at each buffer return */
+ uint64 distance_sum;
+
+ /* maximum actual pinned_buffers observed during the scan */
+ int16 distance_max;
+
+ /* maximum possible look-ahead distance (max_pinned_buffers) */
+ int16 distance_capacity;
+
+ /* number of waits for a read (for the I/O) */
+ uint64 wait_count;
+
+ /* I/O stats */
+ uint64 io_count; /* number of I/Os */
+ uint64 io_nblocks; /* sum of blocks for all I/Os */
+ uint64 io_in_progress; /* sum of in-progress I/Os */
+} IOStats;
+
+typedef struct TableScanInstrumentation
+{
+ IOStats io;
+} TableScanInstrumentation;
+
+/* merge IO statistics from 'src' into 'dst' */
+static inline void
+AccumulateIOStats(IOStats *dst, IOStats *src)
+{
+ dst->prefetch_count += src->prefetch_count;
+ dst->distance_sum += src->distance_sum;
+ if (src->distance_max > dst->distance_max)
+ dst->distance_max = src->distance_max;
+ if (src->distance_capacity > dst->distance_capacity)
+ dst->distance_capacity = src->distance_capacity;
+ dst->wait_count += src->wait_count;
+ dst->io_count += src->io_count;
+ dst->io_nblocks += src->io_nblocks;
+ dst->io_in_progress += src->io_in_progress;
+}
+
+
/* ---------------------
* Instrumentation information for indexscans (amgettuple and amgetbitmap)
* ---------------------
@@ -79,6 +128,7 @@ typedef struct BitmapHeapScanInstrumentation
{
uint64 exact_pages;
uint64 lossy_pages;
+ TableScanInstrumentation stats;
} BitmapHeapScanInstrumentation;
/*
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
index c9359b29b0f..48995c6d534 100644
--- a/src/include/storage/read_stream.h
+++ b/src/include/storage/read_stream.h
@@ -65,6 +65,7 @@
struct ReadStream;
typedef struct ReadStream ReadStream;
+struct IOStats;
/* for block_range_read_stream_cb */
typedef struct BlockRangeReadStreamPrivate
@@ -103,5 +104,6 @@ extern BlockNumber read_stream_pause(ReadStream *stream);
extern void read_stream_resume(ReadStream *stream);
extern void read_stream_reset(ReadStream *stream);
extern void read_stream_end(ReadStream *stream);
+extern void read_stream_enable_stats(ReadStream *stream, struct IOStats *stats);
#endif /* READ_STREAM_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9e6a39f5608..98b8d78e693 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1279,6 +1279,7 @@ IOContext
IOFuncSelector
IOObject
IOOp
+IOStats
IO_STATUS_BLOCK
IPCompareMethod
ITEM
@@ -3127,6 +3128,7 @@ TableLikeClause
TableSampleClause
TableScanDesc
TableScanDescData
+TableScanInstrumentation
TableSpaceCacheEntry
TableSpaceOpts
TableToProcess
--
2.53.0
[text/x-patch] v14-0003-auto_explain-Add-new-GUC-auto_explain.log_io.patch (3.6K, ../../552ecf6c-e8a2-4183-afc4-eff592674475@vondra.me/4-v14-0003-auto_explain-Add-new-GUC-auto_explain.log_io.patch)
download | inline diff:
From c48bc244cccb241c7618603dc91f924a66321a7c Mon Sep 17 00:00:00 2001
From: test <test>
Date: Tue, 7 Apr 2026 02:04:06 +0200
Subject: [PATCH v14 3/5] auto_explain: Add new GUC auto_explain.log_io
Allows enabling the new EXPLAIN "IO" option for auto_explain.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
contrib/auto_explain/auto_explain.c | 15 +++++++++++++++
doc/src/sgml/auto-explain.sgml | 20 ++++++++++++++++++++
2 files changed, 35 insertions(+)
diff --git a/contrib/auto_explain/auto_explain.c b/contrib/auto_explain/auto_explain.c
index 6ceae1c69ce..9114ced45a7 100644
--- a/contrib/auto_explain/auto_explain.c
+++ b/contrib/auto_explain/auto_explain.c
@@ -38,6 +38,7 @@ static int auto_explain_log_parameter_max_length = -1; /* bytes or -1 */
static bool auto_explain_log_analyze = false;
static bool auto_explain_log_verbose = false;
static bool auto_explain_log_buffers = false;
+static bool auto_explain_log_io = false;
static bool auto_explain_log_wal = false;
static bool auto_explain_log_triggers = false;
static bool auto_explain_log_timing = true;
@@ -203,6 +204,17 @@ _PG_init(void)
NULL,
NULL);
+ DefineCustomBoolVariable("auto_explain.log_io",
+ "Log I/O statistics.",
+ NULL,
+ &auto_explain_log_io,
+ false,
+ PGC_SUSET,
+ 0,
+ NULL,
+ NULL,
+ NULL);
+
DefineCustomBoolVariable("auto_explain.log_wal",
"Log WAL usage.",
NULL,
@@ -343,6 +355,8 @@ explain_ExecutorStart(QueryDesc *queryDesc, int eflags)
queryDesc->instrument_options |= INSTRUMENT_ROWS;
if (auto_explain_log_buffers)
queryDesc->instrument_options |= INSTRUMENT_BUFFERS;
+ if (auto_explain_log_io)
+ queryDesc->instrument_options |= INSTRUMENT_IO;
if (auto_explain_log_wal)
queryDesc->instrument_options |= INSTRUMENT_WAL;
}
@@ -440,6 +454,7 @@ explain_ExecutorEnd(QueryDesc *queryDesc)
es->analyze = (queryDesc->instrument_options && auto_explain_log_analyze);
es->verbose = auto_explain_log_verbose;
es->buffers = (es->analyze && auto_explain_log_buffers);
+ es->io = (es->analyze && auto_explain_log_io);
es->wal = (es->analyze && auto_explain_log_wal);
es->timing = (es->analyze && auto_explain_log_timing);
es->summary = es->analyze;
diff --git a/doc/src/sgml/auto-explain.sgml b/doc/src/sgml/auto-explain.sgml
index ee85a67eb2e..06a8fcc6c5b 100644
--- a/doc/src/sgml/auto-explain.sgml
+++ b/doc/src/sgml/auto-explain.sgml
@@ -128,6 +128,26 @@ LOAD 'auto_explain';
</listitem>
</varlistentry>
+ <varlistentry id="auto-explain-configuration-parameters-log-io">
+ <term>
+ <varname>auto_explain.log_io</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>auto_explain.log_io</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ <varname>auto_explain.log_io</varname> controls whether I/O usage
+ statistics are printed when an execution plan is logged; it's
+ equivalent to the <literal>IO</literal> option of <command>EXPLAIN</command>.
+ This parameter has no effect
+ unless <varname>auto_explain.log_analyze</varname> is enabled.
+ This parameter is off by default.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="auto-explain-configuration-parameters-log-wal">
<term>
<varname>auto_explain.log_wal</varname> (<type>boolean</type>)
--
2.53.0
[text/x-patch] v14-0004-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch (14.8K, ../../552ecf6c-e8a2-4183-afc4-eff592674475@vondra.me/5-v14-0004-Add-EXPLAIN-IO-instrumentation-for-SeqScan.patch)
download | inline diff:
From 63ae878a433c5e14ce0c94822832d657ec404796 Mon Sep 17 00:00:00 2001
From: test <test>
Date: Tue, 7 Apr 2026 01:38:53 +0200
Subject: [PATCH v14 4/5] Add EXPLAIN (IO) instrumentation for SeqScan
Adds support for EXPLAIN (IO) instrumentation for sequential scans. This
requires adding shared instrumentation for parallel scans, using the
separate DSM approach introduced by dd78e69cfc33.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/backend/commands/explain.c | 25 +++++
src/backend/executor/execParallel.c | 11 ++
src/backend/executor/nodeSeqscan.c | 133 +++++++++++++++++++++++--
src/include/executor/instrument_node.h | 19 ++++
src/include/executor/nodeSeqscan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/test/regress/expected/explain.out | 18 +++-
src/test/regress/sql/explain.sql | 4 +-
src/tools/pgindent/typedefs.list | 2 +
9 files changed, 209 insertions(+), 13 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ca3071f4ca2..c1928713b52 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2032,6 +2032,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
planstate, es);
if (IsA(plan, CteScan))
show_ctescan_info(castNode(CteScanState, planstate), es);
+ show_scan_io_usage((ScanState *) planstate, es);
break;
case T_Gather:
{
@@ -4102,6 +4103,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_SeqScan:
+ {
+ SharedSeqScanInstrumentation *sinstrument
+ = ((SeqScanState *) planstate)->sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ SeqScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 1a5ec0c305f..9690f0938ae 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -257,6 +257,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecSeqScanEstimate((SeqScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentEstimate((SeqScanState *) planstate,
+ e->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -500,6 +503,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeDSM((SeqScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitDSM((SeqScanState *) planstate,
+ d->pcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
@@ -1148,6 +1154,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_BitmapHeapScanState:
ExecBitmapHeapRetrieveInstrumentation((BitmapHeapScanState *) planstate);
break;
+ case T_SeqScanState:
+ ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
+ break;
default:
break;
}
@@ -1388,6 +1397,8 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
case T_SeqScanState:
if (planstate->plan->parallel_aware)
ExecSeqScanInitializeWorker((SeqScanState *) planstate, pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecSeqScanInstrumentInitWorker((SeqScanState *) planstate, pwcxt);
break;
case T_IndexScanState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeSeqscan.c b/src/backend/executor/nodeSeqscan.c
index 04803b0e37d..5bcb0a861d7 100644
--- a/src/backend/executor/nodeSeqscan.c
+++ b/src/backend/executor/nodeSeqscan.c
@@ -29,6 +29,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
+#include "executor/execParallel.h"
#include "executor/execScan.h"
#include "executor/executor.h"
#include "executor/nodeSeqscan.h"
@@ -65,15 +66,21 @@ SeqNext(SeqScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
/*
* We reach here if the scan is not parallel, or if we're serially
* executing a scan that was planned to be parallel.
*/
scandesc = table_beginscan(node->ss.ss_currentRelation,
estate->es_snapshot,
- 0, NULL,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ 0, NULL, flags);
node->ss.ss_currentScanDesc = scandesc;
}
@@ -302,6 +309,22 @@ ExecEndSeqScan(SeqScanState *node)
*/
scanDesc = node->ss.ss_currentScanDesc;
+ /*
+ * Collect I/O stats for this process into shared instrumentation.
+ */
+ if (node->sinstrument != NULL && IsParallelWorker())
+ {
+ SeqScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->sinstrument->num_workers);
+ si = &node->sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (scanDesc && scanDesc->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io, &scanDesc->rs_instrument->io);
+ }
+ }
+
/*
* close heap scan
*/
@@ -370,6 +393,13 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->pscan_len);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -378,9 +408,7 @@ ExecSeqScanInitializeDSM(SeqScanState *node,
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
}
/* ----------------------------------------------------------------
@@ -410,10 +438,97 @@ ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
- table_beginscan_parallel(node->ss.ss_currentRelation, pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ table_beginscan_parallel(node->ss.ss_currentRelation, pscan, flags);
+}
+
+/*
+ * Compute the amount of space we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecSeqScanInstrumentEstimate(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ Size size;
+
+ if ((estate->es_instrument & INSTRUMENT_IO) == 0 || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedSeqScanInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel sequential scan instrumentation.
+ */
+void
+ExecSeqScanInstrumentInitDSM(SeqScanState *node, ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedSeqScanInstrumentation *sinstrument;
+ Size size;
+
+ if ((estate->es_instrument & INSTRUMENT_IO) == 0 || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedSeqScanInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(SeqScanInstrumentation)));
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->sinstrument = sinstrument;
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if ((estate->es_instrument & INSTRUMENT_IO) == 0)
+ return;
+
+ node->sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
+}
+
+/*
+ * Transfer sequential scan instrumentation from DSM to private memory.
+ */
+void
+ExecSeqScanRetrieveInstrumentation(SeqScanState *node)
+{
+ SharedSeqScanInstrumentation *sinstrument = node->sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedSeqScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(SeqScanInstrumentation);
+
+ node->sinstrument = palloc(size);
+ memcpy(node->sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 22a75ccd863..003dc262b5d 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -266,4 +266,23 @@ typedef struct SharedIncrementalSortInfo
IncrementalSortInfo sinfo[FLEXIBLE_ARRAY_MEMBER];
} SharedIncrementalSortInfo;
+
+/* ---------------------
+ * Instrumentation information for sequential scans
+ * ---------------------
+ */
+typedef struct SeqScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} SeqScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedSeqScanInstrumentation
+{
+ int num_workers;
+ SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedSeqScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeSeqscan.h b/src/include/executor/nodeSeqscan.h
index 7a1490596fb..9c0ad4879d7 100644
--- a/src/include/executor/nodeSeqscan.h
+++ b/src/include/executor/nodeSeqscan.h
@@ -28,4 +28,13 @@ extern void ExecSeqScanReInitializeDSM(SeqScanState *node, ParallelContext *pcxt
extern void ExecSeqScanInitializeWorker(SeqScanState *node,
ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecSeqScanInstrumentEstimate(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitDSM(SeqScanState *node,
+ ParallelContext *pcxt);
+extern void ExecSeqScanInstrumentInitWorker(SeqScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecSeqScanRetrieveInstrumentation(SeqScanState *node);
+
#endif /* NODESEQSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 3ecae7552fc..56febb3204c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1670,6 +1670,7 @@ typedef struct SeqScanState
{
ScanState ss; /* its first field is NodeTag */
Size pscan_len; /* size of parallel heap scan descriptor */
+ struct SharedSeqScanInstrumentation *sinstrument;
} SeqScanState;
/* ----------------
diff --git a/src/test/regress/expected/explain.out b/src/test/regress/expected/explain.out
index dc31c7ce9f9..74a4d87801e 100644
--- a/src/test/regress/expected/explain.out
+++ b/src/test/regress/expected/explain.out
@@ -100,7 +100,7 @@ select explain_filter('explain (buffers, format text) select * from int8_tbl i8'
(1 row)
\a
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
explain_filter
<explain xmlns="http://www.postgresql.org/N/explain">
<Query>
@@ -119,6 +119,13 @@ explain_filter
<Actual-Rows>N.N</Actual-Rows>
<Actual-Loops>N</Actual-Loops>
<Disabled>false</Disabled>
+ <Average-Prefetch-Distance>N.N</Average-Prefetch-Distance>
+ <Max-Prefetch-Distance>N</Max-Prefetch-Distance>
+ <Prefetch-Capacity>N</Prefetch-Capacity>
+ <I-O-Count>N</I-O-Count>
+ <I-O-Waits>N</I-O-Waits>
+ <Average-I-O-Size>N.N</Average-I-O-Size>
+ <Average-I-Os-In-Progress>N.N</Average-I-Os-In-Progress>
<Shared-Hit-Blocks>N</Shared-Hit-Blocks>
<Shared-Read-Blocks>N</Shared-Read-Blocks>
<Shared-Dirtied-Blocks>N</Shared-Dirtied-Blocks>
@@ -149,7 +156,7 @@ explain_filter
</Query>
</explain>
(1 row)
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
explain_filter
- Plan:
Node Type: "Seq Scan"
@@ -166,6 +173,13 @@ explain_filter
Actual Rows: N.N
Actual Loops: N
Disabled: false
+ Average Prefetch Distance: N.N
+ Max Prefetch Distance: N
+ Prefetch Capacity: N
+ I/O Count: N
+ I/O Waits: N
+ Average I/O Size: N.N
+ Average I/Os In Progress: N.N
Shared Hit Blocks: N
Shared Read Blocks: N
Shared Dirtied Blocks: N
diff --git a/src/test/regress/sql/explain.sql b/src/test/regress/sql/explain.sql
index 8f10e1aff55..2f163c64bf6 100644
--- a/src/test/regress/sql/explain.sql
+++ b/src/test/regress/sql/explain.sql
@@ -69,8 +69,8 @@ select explain_filter('explain (analyze, buffers, format text) select * from int
select explain_filter('explain (buffers, format text) select * from int8_tbl i8');
\a
-select explain_filter('explain (analyze, buffers, format xml) select * from int8_tbl i8');
-select explain_filter('explain (analyze, serialize, buffers, format yaml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, buffers, io, format xml) select * from int8_tbl i8');
+select explain_filter('explain (analyze, serialize, buffers, io, format yaml) select * from int8_tbl i8');
select explain_filter('explain (buffers, format json) select * from int8_tbl i8');
\a
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 98b8d78e693..c0436a13ac3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2800,6 +2800,7 @@ SelfJoinCandidate
SemTPadded
SemiAntiJoinFactors
SeqScan
+SeqScanInstrumentation
SeqScanState
SeqTable
SeqTableData
@@ -2864,6 +2865,7 @@ SharedMemoizeInfo
SharedRecordTableEntry
SharedRecordTableKey
SharedRecordTypmodRegistry
+SharedSeqScanInstrumentation
SharedSortInfo
SharedTuplestore
SharedTuplestoreAccessor
--
2.53.0
[text/x-patch] v14-0005-Add-EXPLAIN-IO-instrumentation-for-TidRangeScan.patch (12.5K, ../../552ecf6c-e8a2-4183-afc4-eff592674475@vondra.me/6-v14-0005-Add-EXPLAIN-IO-instrumentation-for-TidRangeScan.patch)
download | inline diff:
From ac6329aaa2e2045dab1804ad09506f8b39906167 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 7 Apr 2026 16:45:11 +0200
Subject: [PATCH v14 5/5] Add EXPLAIN (IO) instrumentation for TidRangeScan
Adds support for EXPLAIN (IO) instrumentation for TidRange scans. This
requires adding shared instrumentation for parallel scans, using the
separate DSM approach introduced by dd78e69cfc33.
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/flat/a177a6dd-240b-455a-8f25-aca0b1c08c6e%40vondra.me
---
src/backend/commands/explain.c | 25 +++++
src/backend/executor/execParallel.c | 12 +++
src/backend/executor/nodeTidrangescan.c | 133 ++++++++++++++++++++++--
src/include/executor/instrument_node.h | 18 ++++
src/include/executor/nodeTidrangescan.h | 9 ++
src/include/nodes/execnodes.h | 1 +
src/tools/pgindent/typedefs.list | 2 +
7 files changed, 192 insertions(+), 8 deletions(-)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index c1928713b52..f9d79879fda 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -2149,6 +2149,7 @@ ExplainNode(PlanState *planstate, List *ancestors,
if (plan->qual)
show_instrumentation_count("Rows Removed by Filter", 1,
planstate, es);
+ show_scan_io_usage((ScanState *) planstate, es);
}
break;
case T_ForeignScan:
@@ -4127,6 +4128,30 @@ show_scan_io_usage(ScanState *planstate, ExplainState *es)
}
}
+ break;
+ }
+ case T_TidRangeScan:
+ {
+ SharedTidRangeScanInstrumentation *sinstrument
+ = ((TidRangeScanState *) planstate)->trss_sinstrument;
+
+ if (sinstrument)
+ {
+ for (int i = 0; i < sinstrument->num_workers; ++i)
+ {
+ TidRangeScanInstrumentation *winstrument = &sinstrument->sinstrument[i];
+
+ AccumulateIOStats(&stats, &winstrument->stats.io);
+
+ if (!es->workers_state)
+ continue;
+
+ ExplainOpenWorker(i, es);
+ print_io_usage(es, &winstrument->stats.io);
+ ExplainCloseWorker(i, es);
+ }
+ }
+
break;
}
default:
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 9690f0938ae..81b87d82fab 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -291,6 +291,9 @@ ExecParallelEstimate(PlanState *planstate, ExecParallelEstimateContext *e)
if (planstate->plan->parallel_aware)
ExecTidRangeScanEstimate((TidRangeScanState *) planstate,
e->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentEstimate((TidRangeScanState *) planstate,
+ e->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -536,6 +539,9 @@ ExecParallelInitializeDSM(PlanState *planstate,
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeDSM((TidRangeScanState *) planstate,
d->pcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitDSM((TidRangeScanState *) planstate,
+ d->pcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
@@ -1157,6 +1163,9 @@ ExecParallelRetrieveInstrumentation(PlanState *planstate,
case T_SeqScanState:
ExecSeqScanRetrieveInstrumentation((SeqScanState *) planstate);
break;
+ case T_TidRangeScanState:
+ ExecTidRangeScanRetrieveInstrumentation((TidRangeScanState *) planstate);
+ break;
default:
break;
}
@@ -1430,6 +1439,9 @@ ExecParallelInitializeWorker(PlanState *planstate, ParallelWorkerContext *pwcxt)
if (planstate->plan->parallel_aware)
ExecTidRangeScanInitializeWorker((TidRangeScanState *) planstate,
pwcxt);
+ /* even when not parallel-aware, for EXPLAIN ANALYZE */
+ ExecTidRangeScanInstrumentInitWorker((TidRangeScanState *) planstate,
+ pwcxt);
break;
case T_AppendState:
if (planstate->plan->parallel_aware)
diff --git a/src/backend/executor/nodeTidrangescan.c b/src/backend/executor/nodeTidrangescan.c
index 4a8fe91b2b3..b387ed6c308 100644
--- a/src/backend/executor/nodeTidrangescan.c
+++ b/src/backend/executor/nodeTidrangescan.c
@@ -18,7 +18,9 @@
#include "access/sysattr.h"
#include "access/tableam.h"
#include "catalog/pg_operator.h"
+#include "executor/execParallel.h"
#include "executor/executor.h"
+#include "executor/instrument.h"
#include "executor/nodeTidrangescan.h"
#include "nodes/nodeFuncs.h"
#include "utils/rel.h"
@@ -242,12 +244,19 @@ TidRangeNext(TidRangeScanState *node)
if (scandesc == NULL)
{
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
+
scandesc = table_beginscan_tidrange(node->ss.ss_currentRelation,
estate->es_snapshot,
&node->trss_mintid,
&node->trss_maxtid,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ flags);
node->ss.ss_currentScanDesc = scandesc;
}
else
@@ -342,6 +351,20 @@ ExecEndTidRangeScan(TidRangeScanState *node)
{
TableScanDesc scan = node->ss.ss_currentScanDesc;
+ /* Collect IO stats for this process into shared instrumentation */
+ if (node->trss_sinstrument != NULL && IsParallelWorker())
+ {
+ TidRangeScanInstrumentation *si;
+
+ Assert(ParallelWorkerNumber < node->trss_sinstrument->num_workers);
+ si = &node->trss_sinstrument->sinstrument[ParallelWorkerNumber];
+
+ if (scan && scan->rs_instrument)
+ {
+ AccumulateIOStats(&si->stats.io, &scan->rs_instrument->io);
+ }
+ }
+
if (scan != NULL)
table_endscan(scan);
}
@@ -454,6 +477,13 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
{
EState *estate = node->ss.ps.state;
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (estate->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_allocate(pcxt->toc, node->trss_pscanlen);
table_parallelscan_initialize(node->ss.ss_currentRelation,
@@ -462,9 +492,7 @@ ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt)
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pscan);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
}
/* ----------------------------------------------------------------
@@ -494,11 +522,100 @@ ExecTidRangeScanInitializeWorker(TidRangeScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelTableScanDesc pscan;
+ uint32 flags = SO_NONE;
+
+ if (ScanRelIsReadOnly(&node->ss))
+ flags |= SO_HINT_REL_READ_ONLY;
+
+ if (node->ss.ps.state->es_instrument & INSTRUMENT_IO)
+ flags |= SO_SCAN_INSTRUMENT;
pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->ss.ss_currentScanDesc =
table_beginscan_parallel_tidrange(node->ss.ss_currentRelation,
- pscan,
- ScanRelIsReadOnly(&node->ss) ?
- SO_HINT_REL_READ_ONLY : SO_NONE);
+ pscan, flags);
+}
+
+/*
+ * Compute the amount of space we'll need for the shared instrumentation and
+ * inform pcxt->estimator.
+ */
+void
+ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ Size size;
+
+ if ((estate->es_instrument & INSTRUMENT_IO) == 0 || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedTidRangeScanInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+
+ shm_toc_estimate_chunk(&pcxt->estimator, size);
+ shm_toc_estimate_keys(&pcxt->estimator, 1);
+}
+
+/*
+ * Set up parallel scan instrumentation.
+ */
+void
+ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt)
+{
+ EState *estate = node->ss.ps.state;
+ SharedTidRangeScanInstrumentation *sinstrument;
+ Size size;
+
+ if ((estate->es_instrument & INSTRUMENT_IO) == 0 || pcxt->nworkers == 0)
+ return;
+
+ size = add_size(offsetof(SharedTidRangeScanInstrumentation, sinstrument),
+ mul_size(pcxt->nworkers, sizeof(TidRangeScanInstrumentation)));
+ sinstrument = shm_toc_allocate(pcxt->toc, size);
+ memset(sinstrument, 0, size);
+ sinstrument->num_workers = pcxt->nworkers;
+ shm_toc_insert(pcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ sinstrument);
+ node->trss_sinstrument = sinstrument;
+}
+
+/*
+ * Look up and save the location of the shared instrumentation.
+ */
+void
+ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt)
+{
+ EState *estate = node->ss.ps.state;
+
+ if ((estate->es_instrument & INSTRUMENT_IO) == 0)
+ return;
+
+ node->trss_sinstrument = shm_toc_lookup(pwcxt->toc,
+ node->ss.ps.plan->plan_node_id +
+ PARALLEL_KEY_SCAN_INSTRUMENT_OFFSET,
+ false);
+}
+
+/*
+ * Transfer scan instrumentation from DSM to private memory.
+ */
+void
+ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node)
+{
+ SharedTidRangeScanInstrumentation *sinstrument = node->trss_sinstrument;
+ Size size;
+
+ if (sinstrument == NULL)
+ return;
+
+ size = offsetof(SharedTidRangeScanInstrumentation, sinstrument)
+ + sinstrument->num_workers * sizeof(TidRangeScanInstrumentation);
+
+ node->trss_sinstrument = palloc(size);
+ memcpy(node->trss_sinstrument, sinstrument, size);
}
diff --git a/src/include/executor/instrument_node.h b/src/include/executor/instrument_node.h
index 003dc262b5d..4076990408e 100644
--- a/src/include/executor/instrument_node.h
+++ b/src/include/executor/instrument_node.h
@@ -285,4 +285,22 @@ typedef struct SharedSeqScanInstrumentation
SeqScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
} SharedSeqScanInstrumentation;
+
+/*
+ * Instrumentation information for TID range scans
+ */
+typedef struct TidRangeScanInstrumentation
+{
+ TableScanInstrumentation stats;
+} TidRangeScanInstrumentation;
+
+/*
+ * Shared memory container for per-worker information
+ */
+typedef struct SharedTidRangeScanInstrumentation
+{
+ int num_workers;
+ TidRangeScanInstrumentation sinstrument[FLEXIBLE_ARRAY_MEMBER];
+} SharedTidRangeScanInstrumentation;
+
#endif /* INSTRUMENT_NODE_H */
diff --git a/src/include/executor/nodeTidrangescan.h b/src/include/executor/nodeTidrangescan.h
index 8752d1ea8c4..9e7d0a357bb 100644
--- a/src/include/executor/nodeTidrangescan.h
+++ b/src/include/executor/nodeTidrangescan.h
@@ -28,4 +28,13 @@ extern void ExecTidRangeScanInitializeDSM(TidRangeScanState *node, ParallelConte
extern void ExecTidRangeScanReInitializeDSM(TidRangeScanState *node, ParallelContext *pcxt);
extern void ExecTidRangeScanInitializeWorker(TidRangeScanState *node, ParallelWorkerContext *pwcxt);
+/* instrument support */
+extern void ExecTidRangeScanInstrumentEstimate(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitDSM(TidRangeScanState *node,
+ ParallelContext *pcxt);
+extern void ExecTidRangeScanInstrumentInitWorker(TidRangeScanState *node,
+ ParallelWorkerContext *pwcxt);
+extern void ExecTidRangeScanRetrieveInstrumentation(TidRangeScanState *node);
+
#endif /* NODETIDRANGESCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 56febb3204c..13359180d25 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1922,6 +1922,7 @@ typedef struct TidRangeScanState
ItemPointerData trss_maxtid;
bool trss_inScan;
Size trss_pscanlen;
+ struct SharedTidRangeScanInstrumentation *trss_sinstrument;
} TidRangeScanState;
/* ----------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c0436a13ac3..0cd41106975 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2867,6 +2867,7 @@ SharedRecordTableKey
SharedRecordTypmodRegistry
SharedSeqScanInstrumentation
SharedSortInfo
+SharedTidRangeScanInstrumentation
SharedTuplestore
SharedTuplestoreAccessor
SharedTuplestoreChunk
@@ -3171,6 +3172,7 @@ TidOpExpr
TidPath
TidRangePath
TidRangeScan
+TidRangeScanInstrumentation
TidRangeScanState
TidScan
TidScanState
--
2.53.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 19:42 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 21:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Melanie Plageman @ 2026-04-07 19:42 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Apr 7, 2026 at 1:34 PM Tomas Vondra <tomas@vondra.me> wrote:
>
> On 4/7/26 18:07, Melanie Plageman wrote:
>
> > I see I had bitmapheapscan use
> > if (!node->ss.ps.instrument || pcxt->nworkers == 0)
> > return;
> >
> > while seq scan uses
> > if (!estate->es_instrument || pcxt->nworkers == 0)
> > return;
> >
> > That does seem worth being consistent about. Though the estate one is
> > probably better to use and changing bitmapheapscan in this commit
> > might be noisy... I don't feel strongly either way.
>
> I'm not sure this is just a question of consistency, because BHS may
> need the shared instrumentation even if (es_instrument = 0), no? While
> the seqscan/tidrange scan only need it with EXPLAIN(IO).
>
> So I think the two nodes should check
>
> ((estate->es_instrument & INSTRUMENT_IO) == 0)
Makes sense to me.
I skimmed v14 quite quickly and LGTM.
- Melanie
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
@ 2026-04-07 21:28 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 21:52 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-04-07 21:28 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Lukas Fittl <lukas@fittl.com>; Andres Freund <andres@anarazel.de>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 4/7/26 21:42, Melanie Plageman wrote:
> On Tue, Apr 7, 2026 at 1:34 PM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> On 4/7/26 18:07, Melanie Plageman wrote:
>>
>>> I see I had bitmapheapscan use
>>> if (!node->ss.ps.instrument || pcxt->nworkers == 0)
>>> return;
>>>
>>> while seq scan uses
>>> if (!estate->es_instrument || pcxt->nworkers == 0)
>>> return;
>>>
>>> That does seem worth being consistent about. Though the estate one is
>>> probably better to use and changing bitmapheapscan in this commit
>>> might be noisy... I don't feel strongly either way.
>>
>> I'm not sure this is just a question of consistency, because BHS may
>> need the shared instrumentation even if (es_instrument = 0), no? While
>> the seqscan/tidrange scan only need it with EXPLAIN(IO).
>>
>> So I think the two nodes should check
>>
>> ((estate->es_instrument & INSTRUMENT_IO) == 0)
>
> Makes sense to me.
>
> I skimmed v14 quite quickly and LGTM.
>
Thanks. I've now pushed all parts of this patch series.
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 21:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-04-07 21:52 ` Andres Freund <andres@anarazel.de>
2026-05-04 13:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-04-07 21:52 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-04-07 23:28:32 +0200, Tomas Vondra wrote:
> Thanks. I've now pushed all parts of this patch series.
Yay.
It'd have been much harder to do the read stream improvements I worked on in
the last weeks without this patchset, and it also was much harder to
understand the performance of the index prefetching patchset ( :( ) before /
without it.
The only thing I don't like about it is not having had it in the 18 cycle,
when working on AIO.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 21:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 21:52 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-05-04 13:51 ` Tomas Vondra <tomas@vondra.me>
2026-05-04 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-05-04 13:51 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Lukas Fittl <lukas@fittl.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
While looking for something in the EXPLAIN docs, I realized the docs
added by 681daed93169 are considerably less detailed that for the other
EXPLAIN options. It only said "information on I/O" provided by the scan
node, while the other options go into much more details.
The attached patch improves that a little bit - it lists the counters
added to explain. I'll consider going into a little bit more detail,
e.g. regarding what "capacity" or "waits" means.
I'll add this as an open item for PG19, so that I don't forget about it.
regards
--
Tomas Vondra
Attachments:
[text/x-patch] 0001-Improve-docs-for-EXPLAIN-IO.patch (1.6K, ../../afa26355-0170-4dfc-a381-024c052d26bf@vondra.me/2-0001-Improve-docs-for-EXPLAIN-IO.patch)
download | inline diff:
From e032752680193a26e21c3927968f72bed20db99e Mon Sep 17 00:00:00 2001
From: test <test>
Date: Sun, 3 May 2026 16:05:54 +0200
Subject: [PATCH] Improve docs for EXPLAIN (IO)
Commit 681daed931 introduced a new EXPLAIN options "IO", but the docs
did not explain what information is added to the output. Expand the
description a little bit, similarly to the other EXPLAIN options.
While at it, fix a typo in the first sentence.
Reported-by: FIXME link to the typo report
Author: Tomas Vondra <tomas@vondra.me>
---
doc/src/sgml/ref/explain.sgml | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/explain.sgml b/doc/src/sgml/ref/explain.sgml
index e95e19081e1..d0644bca038 100644
--- a/doc/src/sgml/ref/explain.sgml
+++ b/doc/src/sgml/ref/explain.sgml
@@ -303,7 +303,13 @@ ROLLBACK;
<term><literal>IO</literal></term>
<listitem>
<para>
- Include information on I/O performed by scan nodes proving such information.
+ Include information on I/O performed by scan nodes providing such
+ information.
+ Specifically, include information about the prefetch queue (the average
+ and maximum distance, and the capacity), and information about issued
+ I/O requests (number of requests and an average request size, number of
+ I/O waits, and an average number of concurrent I/O requests).
+ In text format, only non-zero values are printed.
This parameter may only be used when <literal>ANALYZE</literal> is also
enabled. It defaults to <literal>FALSE</literal>.
</para>
--
2.54.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 21:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 21:52 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-05-04 13:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-05-04 15:22 ` Lukas Fittl <lukas@fittl.com>
2026-05-04 17:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Lukas Fittl @ 2026-05-04 15:22 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, May 4, 2026 at 6:51 AM Tomas Vondra <tomas@vondra.me> wrote:
>
> While looking for something in the EXPLAIN docs, I realized the docs
> added by 681daed93169 are considerably less detailed that for the other
> EXPLAIN options. It only said "information on I/O" provided by the scan
> node, while the other options go into much more details.
>
> The attached patch improves that a little bit - it lists the counters
> added to explain. I'll consider going into a little bit more detail,
> e.g. regarding what "capacity" or "waits" means.
I think that makes sense to add.
Two thoughts:
(1) I think going into a bit more detail makes sense to actually
explain how to interpret the values (or maybe even what action to
take?) - I think the BUFFERS documentation is probably a good style to
follow in terms of how it lists out the individual values and their
meaning.
(2) What do you think of also clarifying when this information is
tracked? I gave that feedback in an earlier patch in this thread, but
I still feel like the "scan nodes providing such information" is very
vague and not helpful for users trying to understand when this is
shown (e.g. the fact that this currently has no visibility on index
I/O, and won't tell you anything about writes being performed to clear
out dirty buffers/etc).
Maybe:
"Include information on I/O performed by scan nodes providing such
information, currently visible for read I/O issued by Bitmap Heap
Scans, Sequential Scans and TID Scans."
Of course that's a bit prone to get stale, but I think the clarity on
what it does/doesn't do would be worth the effort to maintain it.
Thanks,
Lukas
--
Lukas Fittl
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 16:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 21:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 21:52 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-05-04 13:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-05-04 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
@ 2026-05-04 17:16 ` Tomas Vondra <tomas@vondra.me>
0 siblings, 0 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-05-04 17:16 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 5/4/26 17:22, Lukas Fittl wrote:
> On Mon, May 4, 2026 at 6:51 AM Tomas Vondra <tomas@vondra.me> wrote:
>>
>> While looking for something in the EXPLAIN docs, I realized the docs
>> added by 681daed93169 are considerably less detailed that for the other
>> EXPLAIN options. It only said "information on I/O" provided by the scan
>> node, while the other options go into much more details.
>>
>> The attached patch improves that a little bit - it lists the counters
>> added to explain. I'll consider going into a little bit more detail,
>> e.g. regarding what "capacity" or "waits" means.
>
> I think that makes sense to add.
>
> Two thoughts:
>
> (1) I think going into a bit more detail makes sense to actually
> explain how to interpret the values (or maybe even what action to
> take?) - I think the BUFFERS documentation is probably a good style to
> follow in terms of how it lists out the individual values and their
> meaning.
>
Agreed.
> (2) What do you think of also clarifying when this information is
> tracked? I gave that feedback in an earlier patch in this thread, but
> I still feel like the "scan nodes providing such information" is very
> vague and not helpful for users trying to understand when this is
> shown (e.g. the fact that this currently has no visibility on index
> I/O, and won't tell you anything about writes being performed to clear
> out dirty buffers/etc).
>
> Maybe:
>
> "Include information on I/O performed by scan nodes providing such
> information, currently visible for read I/O issued by Bitmap Heap
> Scans, Sequential Scans and TID Scans."
>
> Of course that's a bit prone to get stale, but I think the clarity on
> what it does/doesn't do would be worth the effort to maintain it.
>
Yeah, I'll think about it. At some point I didn't like mentioning exact
scan types, because who knows what custom scans could do etc. But maybe
that's OK.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
@ 2026-04-07 15:09 ` Andres Freund <andres@anarazel.de>
2 siblings, 0 replies; 59+ messages in thread
From: Andres Freund @ 2026-04-07 15:09 UTC (permalink / raw)
To: Lukas Fittl <lukas@fittl.com>; +Cc: Tomas Vondra <tomas@vondra.me>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-04-07 01:00:28 -0700, Lukas Fittl wrote:
> Btw, in that same test, when I re-run I get Prefetch but not I/O -- why is that?
>
> Seq Scan on organizations (cost=0.00..7.42 rows=1 width=483) (actual
> time=0.034..0.042 rows=1.00 loops=1)
> Filter: (slug = 'pganalyze'::text)
> Rows Removed by Filter: 113
> Prefetch: avg=1.00 max=1 capacity=94
> Buffers: shared hit=6
>
> (if its all buffers hit, why are there any prefetches at all?)
It's not really prefetching, which is visible as the distance being 1. The
read stream is created at the start of the scan, not at the first miss.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-31 17:41 ` Andres Freund <andres@anarazel.de>
2026-03-31 18:03 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
1 sibling, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-03-31 17:41 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-03-30 20:21:29 +0200, Tomas Vondra wrote:
> >> From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
> >> From: Tomas Vondra <tomas@vondra.me>
> >> Date: Thu, 19 Mar 2026 22:25:09 +0100
> >> Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
> >>
> >> This adds details about AIO / prefetch for executor nodes using a
> >> ReadStream. Right now this applies only to BitmapHeapScan, because
> >> that's the only scan node using a ReadStream and collecting
> >> instrumentation from workers.
> >
> > I don't understand why that means it should be done as part of this commit,
> > whereas seqscans shouldn't?
> >
>
> Are you suggesting the commit adds support for all those scans (BHS,
> SeqScan and TidRangeScan) or none of them?
I guess I mostly just didn't quite understand what differentiates bitmap scans
from the other scans, based on this explanation.
> To me it seems better to have at least some scan because of testing. But
> SeqScan/TidRangeScan don't have the instrumentation infrastructure for
> parallel queries, and I don't want to do that in the main patch - it seems
> rather unrelated. And I also don't want to add it before the main patch.
I'd probably do the latter, i.e. add it before the main patch. Or at least
separately from the change to show read stream instrumentation.
> > Separately, is %.3f really the right thing? When is that degree of precision
> > useful here?
> >
>
> No idea. I think %.1f would be enough. We use %.2f in a couple places,
> but that's not a huge difference.
I'd probably use .1f here, I don't think you'd ever need more
precision. Showing that it's not a round number is useful, but more than
that... ?
> >> /* ----------------------------------------------------------------
> >> @@ -404,9 +471,50 @@ void
> >> ExecSeqScanInitializeWorker(SeqScanState *node,
> >> ParallelWorkerContext *pwcxt)
> >> {
> >> + EState *estate = node->ss.ps.state;
> >> ParallelTableScanDesc pscan;
> >> + char *ptr;
> >> + Size size;
> >>
> >> pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
> >> node->ss.ss_currentScanDesc =
> >> - table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
> >> + table_beginscan_parallel(node->ss.ss_currentRelation,
> >> + (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
> >> + pscan);
> >> +
> >> + /*
> >> + * Workers don't get the pscan_len value in scan descriptor, so use the
> >> + * TAM callback again. The result has to match the earlier result in
> >> + * ExecSeqScanEstimate.
> >> + */
> >> + size = table_parallelscan_estimate(node->ss.ss_currentRelation,
> >> + estate->es_snapshot);
> >> +
> >
> > That seems quite grotty...
> >
>
> Yes. But what's a better solution?
Think the way you've done it in v6 is better.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 17:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-03-31 18:03 ` Tomas Vondra <tomas@vondra.me>
2026-03-31 18:12 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2026-03-31 18:03 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/31/26 19:41, Andres Freund wrote:
> Hi,
>
> On 2026-03-30 20:21:29 +0200, Tomas Vondra wrote:
>>>> From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
>>>> From: Tomas Vondra <tomas@vondra.me>
>>>> Date: Thu, 19 Mar 2026 22:25:09 +0100
>>>> Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
>>>>
>>>> This adds details about AIO / prefetch for executor nodes using a
>>>> ReadStream. Right now this applies only to BitmapHeapScan, because
>>>> that's the only scan node using a ReadStream and collecting
>>>> instrumentation from workers.
>>>
>>> I don't understand why that means it should be done as part of this commit,
>>> whereas seqscans shouldn't?
>>>
>>
>> Are you suggesting the commit adds support for all those scans (BHS,
>> SeqScan and TidRangeScan) or none of them?
>
> I guess I mostly just didn't quite understand what differentiates bitmap scans
> from the other scans, based on this explanation.
>
>
>> To me it seems better to have at least some scan because of testing. But
>> SeqScan/TidRangeScan don't have the instrumentation infrastructure for
>> parallel queries, and I don't want to do that in the main patch - it seems
>> rather unrelated. And I also don't want to add it before the main patch.
>
> I'd probably do the latter, i.e. add it before the main patch. Or at least
> separately from the change to show read stream instrumentation.
>
Sorry, I'm confused. Which "latter" option you mean?
>
>>> Separately, is %.3f really the right thing? When is that degree of precision
>>> useful here?
>>>
>>
>> No idea. I think %.1f would be enough. We use %.2f in a couple places,
>> but that's not a huge difference.
>
> I'd probably use .1f here, I don't think you'd ever need more
> precision. Showing that it's not a round number is useful, but more than
> that... ?
>
OK, .1f WFM
>
>>>> /* ----------------------------------------------------------------
>>>> @@ -404,9 +471,50 @@ void
>>>> ExecSeqScanInitializeWorker(SeqScanState *node,
>>>> ParallelWorkerContext *pwcxt)
>>>> {
>>>> + EState *estate = node->ss.ps.state;
>>>> ParallelTableScanDesc pscan;
>>>> + char *ptr;
>>>> + Size size;
>>>>
>>>> pscan = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
>>>> node->ss.ss_currentScanDesc =
>>>> - table_beginscan_parallel(node->ss.ss_currentRelation, 0, pscan);
>>>> + table_beginscan_parallel(node->ss.ss_currentRelation,
>>>> + (estate->es_instrument) ? SO_SCAN_INSTRUMENT : 0,
>>>> + pscan);
>>>> +
>>>> + /*
>>>> + * Workers don't get the pscan_len value in scan descriptor, so use the
>>>> + * TAM callback again. The result has to match the earlier result in
>>>> + * ExecSeqScanEstimate.
>>>> + */
>>>> + size = table_parallelscan_estimate(node->ss.ss_currentRelation,
>>>> + estate->es_snapshot);
>>>> +
>>>
>>> That seems quite grotty...
>>>
>>
>> Yes. But what's a better solution?
>
> Think the way you've done it in v6 is better.
>
Thanks, I agree. I'll adjust the table_parallelscan_initialize to take
the length as an extra argument.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 17:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:03 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-03-31 18:12 ` Andres Freund <andres@anarazel.de>
2026-03-31 18:33 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 59+ messages in thread
From: Andres Freund @ 2026-03-31 18:12 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-03-31 20:03:02 +0200, Tomas Vondra wrote:
> On 3/31/26 19:41, Andres Freund wrote:
> > Hi,
> >
> > On 2026-03-30 20:21:29 +0200, Tomas Vondra wrote:
> >>>> From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
> >>>> From: Tomas Vondra <tomas@vondra.me>
> >>>> Date: Thu, 19 Mar 2026 22:25:09 +0100
> >>>> Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
> >>>>
> >>>> This adds details about AIO / prefetch for executor nodes using a
> >>>> ReadStream. Right now this applies only to BitmapHeapScan, because
> >>>> that's the only scan node using a ReadStream and collecting
> >>>> instrumentation from workers.
> >>>
> >>> I don't understand why that means it should be done as part of this commit,
> >>> whereas seqscans shouldn't?
> >>>
> >>
> >> Are you suggesting the commit adds support for all those scans (BHS,
> >> SeqScan and TidRangeScan) or none of them?
> >
> > I guess I mostly just didn't quite understand what differentiates bitmap scans
> > from the other scans, based on this explanation.
> >
> >
> >> To me it seems better to have at least some scan because of testing. But
> >> SeqScan/TidRangeScan don't have the instrumentation infrastructure for
> >> parallel queries, and I don't want to do that in the main patch - it seems
> >> rather unrelated. And I also don't want to add it before the main patch.
> >
> > I'd probably do the latter, i.e. add it before the main patch. Or at least
> > separately from the change to show read stream instrumentation.
> >
>
> Sorry, I'm confused. Which "latter" option you mean?
Adding the instrumentation infrastructure for seqscan, tidscan as a separate
patch before the main commit.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Re: EXPLAIN: showing ReadStream / prefetch stats Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Re: EXPLAIN: showing ReadStream / prefetch stats Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 17:41 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
2026-03-31 18:03 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-31 18:12 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
@ 2026-03-31 18:33 ` Tomas Vondra <tomas@vondra.me>
0 siblings, 0 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-03-31 18:33 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Lukas Fittl <lukas@fittl.com>; Melanie Plageman <melanieplageman@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 3/31/26 20:12, Andres Freund wrote:
> Hi,
>
> On 2026-03-31 20:03:02 +0200, Tomas Vondra wrote:
>> On 3/31/26 19:41, Andres Freund wrote:
>>> Hi,
>>>
>>> On 2026-03-30 20:21:29 +0200, Tomas Vondra wrote:
>>>>>> From 410eaaebe7b814ac9f44c080e153f4ec1d6d6b86 Mon Sep 17 00:00:00 2001
>>>>>> From: Tomas Vondra <tomas@vondra.me>
>>>>>> Date: Thu, 19 Mar 2026 22:25:09 +0100
>>>>>> Subject: [PATCH v5 3/6] explain: show prefetch stats in EXPLAIN (ANALYZE)
>>>>>>
>>>>>> This adds details about AIO / prefetch for executor nodes using a
>>>>>> ReadStream. Right now this applies only to BitmapHeapScan, because
>>>>>> that's the only scan node using a ReadStream and collecting
>>>>>> instrumentation from workers.
>>>>>
>>>>> I don't understand why that means it should be done as part of this commit,
>>>>> whereas seqscans shouldn't?
>>>>>
>>>>
>>>> Are you suggesting the commit adds support for all those scans (BHS,
>>>> SeqScan and TidRangeScan) or none of them?
>>>
>>> I guess I mostly just didn't quite understand what differentiates bitmap scans
>>> from the other scans, based on this explanation.
>>>
>>>
>>>> To me it seems better to have at least some scan because of testing. But
>>>> SeqScan/TidRangeScan don't have the instrumentation infrastructure for
>>>> parallel queries, and I don't want to do that in the main patch - it seems
>>>> rather unrelated. And I also don't want to add it before the main patch.
>>>
>>> I'd probably do the latter, i.e. add it before the main patch. Or at least
>>> separately from the change to show read stream instrumentation.
>>>
>>
>> Sorry, I'm confused. Which "latter" option you mean?
>
> Adding the instrumentation infrastructure for seqscan, tidscan as a separate
> patch before the main commit.
>
I don't like that, because then we're adding instrumentation that
doesn't actually contain anything. How would you even know it does the
right thing? I think it's better to add that after the main commit.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-09-22 15:07 ` Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2 siblings, 1 reply; 59+ messages in thread
From: Nikolay Samokhvalov @ 2026-09-22 15:07 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mar 15, 2026, Tomas Vondra <tomas@vondra.me> wrote:
> The first line "Prefetch" tracks the look-ahead distance, i.e. how many
> blocks ahead the ReadStream is requesting.
> The second line "I/O" is about the I/O requests actually issued - how
> many times we had to wait for the block (when we get to process it),
> average size of a request (in BLCKSZ blocks), and average number of
> in-progress requests.
Our new AI harness for testing found that a rescan can include buffers that
never reach the consumer in the Prefetch average, and prepared the attached
patch. On a clean cluster with io_method=worker, this is a complete
reproducer:
\set ON_ERROR_STOP on
create extension pg_buffercache;
set jit = off;
set max_parallel_workers_per_gather = 0;
set enable_seqscan = off;
set effective_io_concurrency = 16;
create unlogged table reset_t as
select g as id, repeat(md5(g::text), 16) as payload
from generate_series(1, 80000) as g;
vacuum (analyze, freeze) reset_t;
checkpoint;
select pg_buffercache_evict_relation('reset_t');
explain (analyze, buffers, io, timing off, summary off, costs off)
select r.startblock, s.ctid
from (values (0), (300), (600), (900), (1200),
(1500), (1800), (2100), (2400), (2700)) r(startblock)
cross join lateral (
select t.ctid
from reset_t t
where t.ctid >= format('(%s,1)', r.startblock)::tid
and t.ctid < format('(%s,1)', r.startblock + 200)::tid
offset 0 limit 1
) s;
The inner TID Range Scan reports:
Prefetch: avg=1.32 max=2
I/O: count=20 waits=19 size=1.50 in-progress=1.00
Buffers: shared read=30
Each of the ten loops returns one buffer to the consumer at distance one.
A single-loop control reports avg=1.00 max=1.
read_stream_reset() drains unread buffers by calling
read_stream_next_buffer(), which also calls read_stream_count_prefetch().
The nine rescans above add 18 cleanup samples with distance sum 27, so the
reported average is (10 + 27) / (10 + 18) = 1.321428... and max becomes 2.
The attached patch preserves prefetch_count, distance_sum, and distance_max
around that internal drain, while leaving the real I/O statistics
cumulative. It adds a test that checks all three fields with worker and
sync I/O.
On master d39fda1c the test fails without the read_stream.c change and
passes with it; the full test_aio and core regression suites pass. The
same patch applies to REL_19_STABLE b73d13c3, where the reproducer shows
the same avg=1.32 max=2 result and test_aio passes with the fix.
Nik
Attachments:
[application/x-patch] 0001-read-stream-reset-prefetch-stats.patch (5.4K, ../../CAM527d87NihbtZNBCYSu+9cfOvFoZBotSMhY8575dAoKASdvCg@mail.gmail.com/2-0001-read-stream-reset-prefetch-stats.patch)
download | inline diff:
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
index fddf6ef441d814abf3dbbbebfa344ae9d827d678..852cc958560832270b726c3bf620169fbc13eaa8 100644
--- a/src/backend/storage/aio/read_stream.c
+++ b/src/backend/storage/aio/read_stream.c
@@ -1439,6 +1439,9 @@ read_stream_reset(ReadStream *stream)
{
int16 index;
Buffer buffer;
+ uint64 prefetch_count = 0;
+ uint64 distance_sum = 0;
+ int16 distance_max = 0;
/* Stop looking ahead. */
stream->readahead_distance = 0;
@@ -1451,10 +1454,31 @@ read_stream_reset(ReadStream *stream)
stream->buffered_blocknum = InvalidBlockNumber;
stream->fast_path = false;
+ /*
+ * Draining the stream below is an implementation detail, not consumption
+ * by the stream's caller. Preserve the prefetch distance statistics so
+ * that the drained buffers are not counted as having been returned to the
+ * caller. I/O statistics still need to reflect any work done while
+ * draining.
+ */
+ if (stream->stats)
+ {
+ prefetch_count = stream->stats->prefetch_count;
+ distance_sum = stream->stats->distance_sum;
+ distance_max = stream->stats->distance_max;
+ }
+
/* Unpin anything that wasn't consumed. */
while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
ReleaseBuffer(buffer);
+ if (stream->stats)
+ {
+ stream->stats->prefetch_count = prefetch_count;
+ stream->stats->distance_sum = distance_sum;
+ stream->stats->distance_max = distance_max;
+ }
+
/* Unpin any unused forwarded buffers. */
index = stream->next_buffer_index;
while (index < stream->initialized_buffers &&
diff --git a/src/test/modules/test_aio/t/004_read_stream.pl b/src/test/modules/test_aio/t/004_read_stream.pl
index 32311c07ac02bb3ebe21be687dc54bcd5ced0be3..758a36aadc89b99355e4eb41d07652183d68ba11 100644
--- a/src/test/modules/test_aio/t/004_read_stream.pl
+++ b/src/test/modules/test_aio/t/004_read_stream.pl
@@ -115,6 +115,22 @@ sub test_repeated_blocks
}
+sub test_reset_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ $node->safe_psql('postgres', qq/SELECT evict_rel('largeish');/);
+ is(
+ $node->safe_psql(
+ 'postgres',
+ qq/SELECT read_stream_reset_stats('largeish', ARRAY[0, 1, 2, 3]);/),
+ 't',
+ "$io_method: resetting a stream preserves consumer prefetch statistics"
+ );
+}
+
+
sub test_inject_foreign
{
my $io_method = shift;
@@ -267,6 +283,7 @@ sub test_io_method
is($node->safe_psql('postgres', 'SHOW io_method'),
$io_method, "$io_method: io_method set correctly");
+ test_reset_stats($io_method, $node);
test_repeated_blocks($io_method, $node);
SKIP:
diff --git a/src/test/modules/test_aio/test_aio--1.0.sql b/src/test/modules/test_aio/test_aio--1.0.sql
index 762ac29512f7fc1dabfdc47dfe6840bf840832f3..b5948eb8c50b0ca616be15dcc6b1dcbde0b35c41 100644
--- a/src/test/modules/test_aio/test_aio--1.0.sql
+++ b/src/test/modules/test_aio/test_aio--1.0.sql
@@ -64,6 +64,10 @@ CREATE FUNCTION read_stream_for_blocks(rel regclass, blocks int4[], OUT blockoff
RETURNS SETOF record STRICT
AS 'MODULE_PATHNAME' LANGUAGE C;
+CREATE FUNCTION read_stream_reset_stats(rel regclass, blocks int4[])
+RETURNS pg_catalog.bool STRICT
+AS 'MODULE_PATHNAME' LANGUAGE C;
+
/*
* Handle related functions
diff --git a/src/test/modules/test_aio/test_aio.c b/src/test/modules/test_aio/test_aio.c
index 39d857557cfaf7cabe6c9c4b8b0090dec5c5bc78..7594d2efcce4021154a8fa8d4542bed9d2028c2b 100644
--- a/src/test/modules/test_aio/test_aio.c
+++ b/src/test/modules/test_aio/test_aio.c
@@ -20,6 +20,7 @@
#include "access/relation.h"
#include "catalog/pg_type.h"
+#include "executor/instrument_node.h"
#include "fmgr.h"
#include "funcapi.h"
#include "storage/aio.h"
@@ -916,6 +917,62 @@ read_stream_for_blocks(PG_FUNCTION_ARGS)
}
+PG_FUNCTION_INFO_V1(read_stream_reset_stats);
+Datum
+read_stream_reset_stats(PG_FUNCTION_ARGS)
+{
+ Oid relid = PG_GETARG_OID(0);
+ ArrayType *blocksarray = PG_GETARG_ARRAYTYPE_P(1);
+ Relation rel;
+ BlocksReadStreamData stream_data;
+ ReadStream *stream;
+ IOStats stats = {0};
+ Buffer buf;
+ uint64 prefetch_count;
+ uint64 distance_sum;
+ int16 distance_max;
+
+ if (ARR_NDIM(blocksarray) != 1 ||
+ ARR_HASNULL(blocksarray) ||
+ ARR_ELEMTYPE(blocksarray) != INT4OID ||
+ ARR_DIMS(blocksarray)[0] == 0)
+ elog(ERROR, "expected non-empty 1 dimensional int4 array");
+
+ stream_data.curblock = 0;
+ stream_data.nblocks = ARR_DIMS(blocksarray)[0];
+ stream_data.blocks = (uint32 *) ARR_DATA_PTR(blocksarray);
+
+ rel = relation_open(relid, AccessShareLock);
+ stream = read_stream_begin_relation(READ_STREAM_FULL,
+ NULL,
+ rel,
+ MAIN_FORKNUM,
+ read_stream_for_blocks_cb,
+ &stream_data,
+ 0);
+ read_stream_enable_stats(stream, &stats);
+
+ buf = read_stream_next_buffer(stream, NULL);
+ if (!BufferIsValid(buf))
+ elog(ERROR, "first read_stream_next_buffer() call is unexpectedly invalid");
+ ReleaseBuffer(buf);
+
+ if (stats.prefetch_count != 1)
+ elog(ERROR, "expected exactly one prefetch sample");
+ prefetch_count = stats.prefetch_count;
+ distance_sum = stats.distance_sum;
+ distance_max = stats.distance_max;
+
+ read_stream_reset(stream);
+ read_stream_end(stream);
+ relation_close(rel, NoLock);
+
+ PG_RETURN_BOOL(stats.prefetch_count == prefetch_count &&
+ stats.distance_sum == distance_sum &&
+ stats.distance_max == distance_max);
+}
+
+
PG_FUNCTION_INFO_V1(handle_get);
Datum
handle_get(PG_FUNCTION_ARGS)
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 15:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
@ 2026-09-22 15:22 ` Tomas Vondra <tomas@vondra.me>
2026-09-22 15:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Peter Geoghegan <pg@bowt.ie>
2026-09-22 19:02 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
0 siblings, 2 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-09-22 15:22 UTC (permalink / raw)
To: Nikolay Samokhvalov <nik@postgres.ai>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 9/22/26 17:07, Nikolay Samokhvalov wrote:
> On Mar 15, 2026, Tomas Vondra <tomas@vondra.me> wrote:
>> The first line "Prefetch" tracks the look-ahead distance, i.e. how many
>> blocks ahead the ReadStream is requesting.
>> The second line "I/O" is about the I/O requests actually issued - how
>> many times we had to wait for the block (when we get to process it),
>> average size of a request (in BLCKSZ blocks), and average number of
>> in-progress requests.
>
> Our new AI harness for testing found that a rescan can include buffers that
> never reach the consumer in the Prefetch average, and prepared the attached
> patch. On a clean cluster with io_method=worker, this is a complete
> reproducer:
>
> \set ON_ERROR_STOP on
> create extension pg_buffercache;
> set jit = off;
> set max_parallel_workers_per_gather = 0;
> set enable_seqscan = off;
> set effective_io_concurrency = 16;
>
> create unlogged table reset_t as
> select g as id, repeat(md5(g::text), 16) as payload
> from generate_series(1, 80000) as g;
> vacuum (analyze, freeze) reset_t;
> checkpoint;
> select pg_buffercache_evict_relation('reset_t');
>
> explain (analyze, buffers, io, timing off, summary off, costs off)
> select r.startblock, s.ctid
> from (values (0), (300), (600), (900), (1200),
> (1500), (1800), (2100), (2400), (2700)) r(startblock)
> cross join lateral (
> select t.ctid
> from reset_t t
> where t.ctid >= format('(%s,1)', r.startblock)::tid
> and t.ctid < format('(%s,1)', r.startblock + 200)::tid
> offset 0 limit 1
> ) s;
>
> The inner TID Range Scan reports:
>
> Prefetch: avg=1.32 max=2
> I/O: count=20 waits=19 size=1.50 in-progress=1.00
> Buffers: shared read=30
>
> Each of the ten loops returns one buffer to the consumer at distance one.
> A single-loop control reports avg=1.00 max=1.
>
> read_stream_reset() drains unread buffers by calling
> read_stream_next_buffer(), which also calls read_stream_count_prefetch().
> The nine rescans above add 18 cleanup samples with distance sum 27, so the
> reported average is (10 + 27) / (10 + 18) = 1.321428... and max becomes 2.
>
> The attached patch preserves prefetch_count, distance_sum, and distance_max
> around that internal drain, while leaving the real I/O statistics
> cumulative. It adds a test that checks all three fields with worker and
> sync I/O.
>
It's not clear to me why this would be desirable. The query execution
clearly *does* perform the I/O, even if the buffers end up not being
used. Hiding that would be very confusing / misleading.
In other words, I/O and buffers may disagree - that's not a bug. Those
counters are tracking different things.
To handle this correctly, we'd need to find a way to not issue the I/O
at all (since 862092932c9479b79732f3b441da05453ea5e06d we discard I/O
that was not initiated yet).
> On master d39fda1c the test fails without the read_stream.c change and
> passes with it; the full test_aio and core regression suites pass. The
> same patch applies to REL_19_STABLE b73d13c3, where the reproducer shows
> the same avg=1.32 max=2 result and test_aio passes with the fix.
>
> Nik
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 15:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-09-22 15:35 ` Peter Geoghegan <pg@bowt.ie>
2026-09-22 15:38 ` Re: EXPLAIN: showing ReadStream / prefetch stats Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 59+ messages in thread
From: Peter Geoghegan @ 2026-09-22 15:35 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Nikolay Samokhvalov <nik@postgres.ai>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Sep 22, 2026 at 11:23 AM Tomas Vondra <tomas@vondra.me> wrote:
> In other words, I/O and buffers may disagree - that's not a bug. Those
> counters are tracking different things.
>
> To handle this correctly, we'd need to find a way to not issue the I/O
> at all (since 862092932c9479b79732f3b441da05453ea5e06d we discard I/O
> that was not initiated yet).
Isn't this what Andres' experimental "Make read_stream_reset()/end()
not wait for IO" patch does? See:
https://www.postgresql.org/message-id/f3xxfrkafjxpyqxywcxricxgyizjirfceychyxsgn7bwjp5eda@kwbduhy7tfm...
This is one of the patches that I've been posting as part of the index
prefetching patch series for some time now.
--
Peter Geoghegan
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 15:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 15:35 ` Re: EXPLAIN: showing ReadStream / prefetch stats Peter Geoghegan <pg@bowt.ie>
@ 2026-09-22 15:38 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 59+ messages in thread
From: Andres Freund @ 2026-09-22 15:38 UTC (permalink / raw)
To: Peter Geoghegan <pg@bowt.ie>; +Cc: Tomas Vondra <tomas@vondra.me>; Nikolay Samokhvalov <nik@postgres.ai>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
On 2026-09-22 11:35:29 -0400, Peter Geoghegan wrote:
> On Tue, Sep 22, 2026 at 11:23 AM Tomas Vondra <tomas@vondra.me> wrote:
> > In other words, I/O and buffers may disagree - that's not a bug. Those
> > counters are tracking different things.
+1
> > To handle this correctly, we'd need to find a way to not issue the I/O
> > at all (since 862092932c9479b79732f3b441da05453ea5e06d we discard I/O
> > that was not initiated yet).
>
> Isn't this what Andres' experimental "Make read_stream_reset()/end()
> not wait for IO" patch does? See:
>
> https://www.postgresql.org/message-id/f3xxfrkafjxpyqxywcxricxgyizjirfceychyxsgn7bwjp5eda@kwbduhy7tfm...
>
> This is one of the patches that I've been posting as part of the index
> prefetching patch series for some time now.
They're still issued in that patch, we just don't wait for them anymore. But I
think even so, they should still be accounted for.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 15:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
@ 2026-09-22 19:02 ` Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 19:13 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
1 sibling, 1 reply; 59+ messages in thread
From: Nikolay Samokhvalov @ 2026-09-22 19:02 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Sep 22, 2026 at 8:23 AM Tomas Vondra <tomas@vondra.me> wrote:
> It's not clear to me why this would be desirable. The query execution
> clearly *does* perform the I/O, even if the buffers end up not being
> used. Hiding that would be very confusing / misleading.
>
> In other words, I/O and buffers may disagree - that's not a bug. Those
> counters are tracking different things.
Thanks all. Agreed that issued I/O should remain accounted for.
Is Prefetch avg/max also intended to include buffers discarded by
read_stream_reset()? If so, agreed, the patch is wrong. The comment above
read_stream_count_prefetch() says it is called once per buffer returned
to the consumer.
Nik
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: EXPLAIN: showing ReadStream / prefetch stats
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 15:07 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 15:22 ` Re: EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-09-22 19:02 ` Re: EXPLAIN: showing ReadStream / prefetch stats Nikolay Samokhvalov <nik@postgres.ai>
@ 2026-09-22 19:13 ` Tomas Vondra <tomas@vondra.me>
0 siblings, 0 replies; 59+ messages in thread
From: Tomas Vondra @ 2026-09-22 19:13 UTC (permalink / raw)
To: Nikolay Samokhvalov <nik@postgres.ai>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 9/22/26 21:02, Nikolay Samokhvalov wrote:
> On Tue, Sep 22, 2026 at 8:23 AM Tomas Vondra <tomas@vondra.me> wrote:
>> It's not clear to me why this would be desirable. The query execution
>> clearly *does* perform the I/O, even if the buffers end up not being
>> used. Hiding that would be very confusing / misleading.
>>
>> In other words, I/O and buffers may disagree - that's not a bug. Those
>> counters are tracking different things.
>
> Thanks all. Agreed that issued I/O should remain accounted for.
>
> Is Prefetch avg/max also intended to include buffers discarded by
> read_stream_reset()? If so, agreed, the patch is wrong. The comment above
> read_stream_count_prefetch() says it is called once per buffer returned
> to the consumer.
>
Yeah, that's a good point. The comment is misleading. I didn't consider
that read_stream_next_buffer() is called from _reset() when I wrote it.
Will think how to clarify it.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 59+ messages in thread
end of thread, other threads:[~2026-09-22 19:13 UTC | newest]
Thread overview: 59+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-03-15 19:49 EXPLAIN: showing ReadStream / prefetch stats Tomas Vondra <tomas@vondra.me>
2026-03-16 07:07 ` Lukas Fittl <lukas@fittl.com>
2026-03-16 17:29 ` Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 18:26 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 00:15 ` Tomas Vondra <tomas@vondra.me>
2026-03-16 17:14 ` Melanie Plageman <melanieplageman@gmail.com>
2026-03-16 17:49 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 17:51 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 18:40 ` Andres Freund <andres@anarazel.de>
2026-03-17 19:32 ` Tomas Vondra <tomas@vondra.me>
2026-03-17 20:49 ` Andres Freund <andres@anarazel.de>
2026-03-17 21:43 ` Tomas Vondra <tomas@vondra.me>
2026-03-18 22:41 ` Tomas Vondra <tomas@vondra.me>
2026-03-19 18:11 ` Lukas Fittl <lukas@fittl.com>
2026-03-19 20:57 ` Tomas Vondra <tomas@vondra.me>
2026-03-26 00:35 ` Tomas Vondra <tomas@vondra.me>
2026-03-30 17:28 ` Andres Freund <andres@anarazel.de>
2026-03-30 18:21 ` Tomas Vondra <tomas@vondra.me>
2026-03-31 13:16 ` Tomas Vondra <tomas@vondra.me>
2026-03-31 18:10 ` Andres Freund <andres@anarazel.de>
2026-03-31 18:57 ` Tomas Vondra <tomas@vondra.me>
2026-04-02 15:05 ` Tomas Vondra <tomas@vondra.me>
2026-04-03 19:01 ` Tomas Vondra <tomas@vondra.me>
2026-04-05 22:18 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-05 22:46 ` Tomas Vondra <tomas@vondra.me>
2026-04-05 22:50 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 09:14 ` Tomas Vondra <tomas@vondra.me>
2026-04-06 14:01 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 16:50 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:33 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-06 20:39 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 00:19 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 01:14 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 08:00 ` Lukas Fittl <lukas@fittl.com>
2026-04-07 08:05 ` Lukas Fittl <lukas@fittl.com>
2026-04-07 09:32 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 12:36 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 15:27 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 16:13 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 18:12 ` Andres Freund <andres@anarazel.de>
2026-04-07 16:07 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 17:34 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 19:42 ` Melanie Plageman <melanieplageman@gmail.com>
2026-04-07 21:28 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 21:52 ` Andres Freund <andres@anarazel.de>
2026-05-04 13:51 ` Tomas Vondra <tomas@vondra.me>
2026-05-04 15:22 ` Lukas Fittl <lukas@fittl.com>
2026-05-04 17:16 ` Tomas Vondra <tomas@vondra.me>
2026-04-07 15:09 ` Andres Freund <andres@anarazel.de>
2026-03-31 17:41 ` Andres Freund <andres@anarazel.de>
2026-03-31 18:03 ` Tomas Vondra <tomas@vondra.me>
2026-03-31 18:12 ` Andres Freund <andres@anarazel.de>
2026-03-31 18:33 ` Tomas Vondra <tomas@vondra.me>
2026-09-22 15:07 ` Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 15:22 ` Tomas Vondra <tomas@vondra.me>
2026-09-22 15:35 ` Peter Geoghegan <pg@bowt.ie>
2026-09-22 15:38 ` Andres Freund <andres@anarazel.de>
2026-09-22 19:02 ` Nikolay Samokhvalov <nik@postgres.ai>
2026-09-22 19:13 ` Tomas Vondra <tomas@vondra.me>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox